Checkpointing in Newbler

[Mona]

Hello Everyone,

I have a question, may be too simple but I don't know how to do that. I had been running Newbler on the data having reads from 454 and Illumina paired end sequencing. Because of some reason i had to restart the machine and stop the assembly in the middle. Now I want to resume that again from the last checkpoint, but don't know how to do that. Does anyone have any suggestions for that?

[yaximik]

I doubt you can resume. Also I did nit try to stop assembly or mapping, several times v.2.7 programs exited with error after computations were essentially complete as all files were present in relevant folders. Yet the assembly or mapping projects could not be opened and considered corrupted after restarting the program. Files can be opened and examined by other programs. There are a whole bunch of invisible files in the result folder with all kinds of metadata, which are unusable if programs did not exit normally.

[Mona]

I read it in the blog written by Lex Nederbragt about Newbler:

http://contig.wordpress.com/about/

From the following part of the blog, I assume its possible to resume assembly if stopped anyway:

"After long overlap follows short overlap:

Setting up overlap detection…
XXXXXXX reads to align
Building a tree for YYYYYYYY seeds…
Computing alignments…

Then the curious message appears:

Checkpointing…

Basically, checkpointing means writing the intermediate results to disc, so that in the case of a crash, you could continue the assembly from the last ‘checkpoint’. "

[yaximik]

Although I did not read the blog yet (would be interesting though), such role of checkpointing makes a little sense to me. The most computing-intensive job, aligning reads, happens before that. If program crashes at this phase, like running out of memory or whatever, no resume can be done as checkpointing have not started yet. In my crash cases, programs crashed after checkpointing is done, even after the generating output phase, but the whole project still remains corrupted as result.

[Mona]

For me it crashed at the end when it was detangling allignments, thats y i was looking for a way to use the already aligned reads and finishing the assembly step.but seems i have to start that again.

[flxlex]

You could try 'cd' into the folder that newbler created. If you do

ls -la assembly/

you'll see a bunch of hidden files. You can try to restart the assembly with the runProject command. See http://contig.wordpress.com/2010/06/...novo-assembly/

Repeat the parameters you gave to runAssembly that are appropriate (use runProject --help to check which ones you can use).

Good luck!