Newbler Error: no progress

[DNAjunk]

Hi

The Newbler program (gsAssembler) reports me the following error messages when run via the GUI:

Error myProject: Process appears stalled.
myProject: No progress has been reported for over 10 minutes.
Error myProject: Unable to launch back-end program to read alignment results.
myProject: This normally occurs if the project is in use by another instance of the program,
myProject: or if a computation on the project is being run from the command line.
myProject: If that is not the case, then some data indices maybe in an inconsistent state.
myProject: Performing a full (non-incremental) analysis may remedy the situation.
myProject: If not, please report this error to your customer support representative

When I execute Newbler (runAssembly) via the command line interface, then the program is running everlasting without any further output than the one shown below, no error message is reported:

[home] /apps/bi/454/bin/runProject myProject/
Assembly computation starting at: Mon Aug 31 13:52:11 2009 (v2.0.00.22)
Indexing 001.sff...
-> 439649 reads, 128476283 bases.
Warning: Suspected 3' primer AAAAAAA, 1828 exact matches found.
Indexing 002.sff...
-> 441892 reads, 142281750 bases.
Warning: Suspected 3' primer AAAAAAA, 1168 exact matches found.
Indexing 003.sff...
-> 409573 reads, 128655780 bases.
Indexing 004.sff...
-> 458442 reads, 153830745 bases.
Indexing 005.sff...
-> 471970 reads, 157889257 bases.
Warning: Suspected 3' primer AAAAAAA, 1059 exact matches found.
Indexing 006.sff...
-> 406361 reads, 120353939 bases.
Indexing 007.sff...
-> 426441 reads, 135241914 bases.
Setting up overlap detection...
-> 3053967 of 3053967
Building a tree for 76985221 seeds...
Computing alignments...
-> 1000 of 3012090

Did anyone else experience a similar situation or has some ideas on this issue?

Thank you!

[westerman]

You are probably working with a large and/or complex genome. Use the '-large' command line option. This will make some simplifying assumptions but should not affect the assembly too adversely. See the manual for details. There may be some corresponding checkbox for the GUI (which I do not use and so am not familiar with it.)

[LLB]

There is a check box for "large or complex genomes" in the GUI.

[flxlex]

Could your GUI problem be related to the information in the Roche Technical Bulletin on the update from 2.0.00 to 2.0.01?

In the case of long running computations, the 2.0.00 GUI was incorrectly concluding that the backend computation was stalled and it would display a warning message and stop tracking the progress of the computation. In the 2.0.01 version, the warning message will only be displayed if no progress has been observed for 3 hours and, more importantly, the GUI will continue to track the progress, regardless. With 2.0.01, the user is kept in control of the situation and must hit the “Stop” button to indicate that the GUI should stop tracking the progress (since the user believes that the backend computation has become non-responsive for some reason). It should be an extremely rare situation for the user to have to hit the “Stop” button for this purpose, but the functionality is required in order to prevent a user from being locked out of a project from the GUI in the case that there is an unexpected from with the backend computation.

If so, you might want to upgrade to 2.0.01...
And yes, I would go for the -large option as well. BTW, the Technical Bulletin mentions that handling of large complex genomes is also improved in version 2.0.01.

[DNAjunk]

Thanks to all for the support.

I envisage an upgrade to the latest version.
Indeed, the data set I am dealing with is large.
So, I use the "-large" option when running from the command line interface, and luckily, Newbler is making progress, sometimes very, very slow though.

Do you know whether parts of the Newbler algorithm could be run in parallel?
I.e., I submit the runProject script to an external cluster via qsub, does it speed up when I use "-l nodes=npn=m" with n,m>1?
Regarding this matter, I haven't found any word in the manual or forum, so, I expect it is not possible. What are your experiences?

[westerman]

Do you know whether parts of the Newbler algorithm could be run in parallel?

The runAnalysisPipe runs in parallel, but then you probably already know that.

I have not found a way to run runAssembly (or runProject which is a subset command) in parallel. But neither seems to take too long (hours, not days) especially with the '-large' option plus proper vector and contaminate trimming and/or masking.