Split mate pair 454 for newbler

[flobpf]

Hi,

I have a bunch of 454 3kb mate pair files in SFF format which I'd like to use to get a Newbler assembly. However, I want Newbler to use only those reads which have a linker sequence. Is it possible to provide such parameters to Newbler?

Thanks

[flxlex]

The short answer is 'no'. Newbler will check each read for the presence of the linker, split the reads that have one, but it uses the non-linker containing reads as shotgun reads.

One way to achieve what you want would be to do an regular newbler assembly, extract IDs of the reads containing the linker from the 454PairStatus.txt (only reads with linkers are mentioned here), put these IDs in a text file, and use the -fi option with this file to have newbler assemble only those reads.

[flobpf]

Hi Flxlex,

Thanks for your response. That is what I'm attempting to do. However, my Newbler run takes a really long time (100 hours, 30gb and still only 4% complete!!). I have 7 plates of 3kb mate pairs.

I was thinking of the following alternate approach. Would that work??
1) Use sff_extract to identify "linkered" sequences
2) Split them into .f and .r based on linkers and quality-clip sequences
3) Generate FASTQ files from only the seq with .f and .r
4) Convert FASTQ to FASTA
4) Use FASTA as input to Newbler.

Would be glad to know if that'd work. Also, is there a way to speed up my Newbler run? I'm using the steps mentioned in your post here and here:


Thanks for your help!

[kmcarr]

Hi Flxlex,

Thanks for your response. That is what I'm attempting to do. However, my Newbler run takes a really long time (100 hours, 30gb and still only 4% complete!!). I have 7 plates of 3kb mate pairs.

I was thinking of the following alternate approach. Would that work??
1) Use sff_extract to identify "linkered" sequences
2) Split them into .f and .r based on linkers and quality-clip sequences
3) Generate FASTQ files from only the seq with .f and .r
4) Convert FASTQ to FASTA
4) Use FASTA as input to Newbler.

Would be glad to know if that'd work. Also, is there a way to speed up my Newbler run? I'm using the steps mentioned in your post here and here:


Thanks for your help!

To me that method seems overly complicated and you would loose one of the advantages of Newbler, namely performing its alignments in "flow-space" vs. "base-space". The major problem now seems to be getting Newbler to perform the first assembly so that you can generate a list of reads with are truly paired to pass to a second Newbler assembly. I would suggest an alternate method of identifying the paired reads.

1. Dump FASTA format sequence files from your SFF files using the Roche sffinfo tool.

2. Using your favorite nucleotide pattern matching program (cross_match, SSAHA2, fuzznuc (EMBOSS)) search the FASTA files for reads containing the PE linker sequence.

3. Save the list of accessions for reads with the PE linker to a text file.

4. Use this text file with the -fi option as described above.

This is really just a modification of the method you are currently trying but using, perhaps, a faster method of identifying the paired reads.

I am a little surprised though by how long Newbler is taking and if a significant fraction of your reads are truly paired (i.e. you won't be eliminating the majority of your input reads) it may still stump Newbler.

[flobpf]

Thanks!

Ah Kevin.

Thanks. Thats actually way simpler. Will do it that way.

[flxlex]

Hi,

If you have an assembly running, you will notice that after the phase where newbler reads all the sequence file, there is the 454ReadStatus.txt file. This file can be used even if the assembly is not yet finished to get to the reads with the linker: these will be marked _left and _right. Saves you from having to do the mapping of the linker yourself...

About the assembly speed: have you tried using more cpus's (with the -cpu flag) and the -large option?

[flobpf]

Solved!

Hi,

If you have an assembly running, you will notice that after the phase where newbler reads all the sequence file, there is the 454ReadStatus.txt file. This file can be used even if the assembly is not yet finished to get to the reads with the linker: these will be marked _left and _right. Saves you from having to do the mapping of the linker yourself...

About the assembly speed: have you tried using more cpus's (with the -cpu flag) and the -large option?

Hi Flxlex,

Thanks for your response. I did provide it with the -cpu option and the -large option. That made all the difference and my assembly got over at a blazing speed.