Installing Newbler 2.7 on Ubuntu 12.04, java problem

[JackieBadger]

I assumed it was because for a human sized genome I needed a lot more RAM, so we are using some clusters.
Cheers,

[sklages]

Have you solved your "50% problem"?

[JackieBadger]

Thanks. WIll do.

The reference is a genome of a closely related species to the cDNA query sequencs I have (poly A/T trimmed).
So I want to use the reference to essential construct ESTs and look for SNPs.
De Novo EST assembly works fine, although it seems intuative to use a reference where possible.

[sklages]

Seems gsmapper wants to use more than 64GB RAM .. well, it's a "heavy" reference ...
Just take a few thousands of the largest contigs (is it a cDNA backbone?) and try again, just to see if the installation itself is OK (it probably is).

Finally you need to reduce your reference contigs or look for a more powerful machine.

Just curious, why do you want to map many, many contigs on even more many, many contigs? Maybe you should look for another software to achieve the results you are expecting?

EDIT: Oh, I haven't noticed your 50% issue. Maybe you should contact your sysadm about that. It is worth a try ;-)

[JackieBadger]

OK so I got the memory crash again on the more powerful machine.

Here is the ulimit -a output:

core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 514997
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 514997
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited


My reference file is a genome scaffold consiting of 2,962,365 contigs; 1,555,444,314 bp

My query sequences consit of 1,151,700 contigs: 390,436,936 bp
It is after the Index/screening of my query sequences where the compuation runs out of memory.
On both machines (29GB, and 62GB available respectivly) mapping crashes when it uses a little over 50% of the available RAM

ANy ideas how to remedy?
Maybe I should cut my refernce file in half?

THanks again, guys

[JackieBadger]

Thanks a lot, guys. Its all up and running now.
ulimit -a shows no limits on the machine on the machine with the new installation.

Thanks again

[sklages]

Maybe it is due to the Java version, java/jdk1.7.0_21?
Any ideas?

Cheers

Yes, it is. Do not use Java 7 with the Roche suite, instead go for 6.

Nevertheless, don't forget to check if there are any limts on your machine,

ulimit -a

should help ..

[JackieBadger]

I edited the start up scripts to point in the direction of the new Oracle Java executable, and it seems to initially work.

The GUI looks to start up, with the picture of the 454 machine and "starting...".... but then dissapears without crashing out in the terminal. So the terminal still indicates the program is running but no GUI.

Maybe it is due to the Java version, java/jdk1.7.0_21?
Any ideas?

Cheers

[JackieBadger]

What system are you using for the real 64GB machine?
Go and get Oracle's 64bit Java 6 SDK/JRE, put it somewhere under /home/mgplab/454/, make it known to the scripts and give it a try .. :-)

Roche's suite is a mixture of 32 and 64bit software, .. one step towards 64bit ;-)

Im using Ubuntu. OK..Ill but the java executables in the 454 tree and take a look at the scripts to edit.
Thanks again

[sklages]

I actually just tracked the memory usage during while gsMapper indexed the reference reads. The program crashes stating it is out of memory at ~50% of the total available RAM. i.e. ~14GB RAM.

I wonder why this is? Is there anyway to open gsMapper telling it to use all 29GBs?

No idea if there could be any issues with the VM itself. Maybe some limits:

ulimit -a

should give you a hint about that.

[sklages]

What system are you using for the real 64GB machine?
Go and get Oracle's 64bit Java 6 SDK/JRE, put it somewhere under /home/mgplab/454/, make it known to the scripts and give it a try .. :-)

Roche's suite is a mixture of 32 and 64bit software, .. one step towards 64bit ;-)

[JackieBadger]

I actually just tracked the memory usage during while gsMapper indexed the reference reads. The program crashes stating it is out of memory at ~50% of the total available RAM. i.e. ~14GB RAM.

I wonder why this is? Is there anyway to open gsMapper telling it to use all 29GBs?

[ajthomas]

In Red Hat/Fedora, it's as simple as "yum install zlib.i386..." I don't use Ubuntu, but as i understand, the installation procedure is similar. This thread says to install ia32-libs to get them. Not having done this on Ubuntu myself, however, I can't offer any more help than that.

[JackieBadger]

OK...

So I managed to get the program working on my own machine, but the assembly maxed out the RAM (~29GB in a virtual machine). As such I am trying to now install on a linux dedicated machine that has 64GB.

But I am running into problems again.

The installer says I lack the libraries zlib:i386 Lib xi:i386 Lib xtst:i386 and LibXaw:i386

I can not remember how I installed these on my personal cpu a couple of years ago and am now struggling to find a source for these. It seems they are now obsolete??

If I continue with installation and ignore these, I get the following error when atempting to run gsMapper. Any solution for a non-linux boffin would be very much appreciated

"[email protected]:~/454/apps/mapper/bin$ ./gsMapper
Exception in thread "main" java.lang.UnsatisfiedLinkError: /home/mgplab/454/apps/jre/jre1.6.0_30/lib/i386/xawt/libmawt.so: libXi.so.6: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.load0(Unknown Source)
at java.lang.System.load(Unknown Source)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
at java.lang.System.loadLibrary(Unknown Source)
at sun.security.action.LoadLibraryAction.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.awt.Toolkit.loadLibraries(Unknown Source)
at java.awt.Toolkit.<clinit>(Unknown Source)
at java.awt.Rectangle.<clinit>(Unknown Source)
at com.fourfivefour.ui.Resources.<clinit>(Resources.java:47)
at com.fourfivefour.ui.ResourceFactory.getNewInstance(ResourceFactory.java:79)
at com.fourfivefour.ui.ResourceFactory.get(ResourceFactory.java:66)
at com.fourfivefour.assembly.MapperMain.<clinit>(MapperMain.java:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
"

[JackieBadger]

You don't want to change your login shell, but the system shell:

http://wiki.ubuntuusers.de/Dash

$ sudo dpkg-reconfigure dash

This is asking you if you want to use 'dash' for '/bin/sh'. Answer 'no' to use 'bash' instead.

hth,
Sven

Thanks!
This worked..and installation completed with no apparent issues.
Appreciate the help guys!