Too many short reads and too little RAM?

[samanta]

Someone asked me whether it makes sense to remove duplicate reads to get the library size down to fit RAM limit. I think it is a bad strategy as explained here -

http://www.homolog.us/blogs/2011/09/20/maximizing-utility-of-available-rams-in-k-mer-world/

[zhidkov.ilia]

I think duplicated reads removed to avoid biases that resulted from library preparation (for example) and not for reduction of data for de-novo assembly.

Ilia

[samanta]

That's a good point. Some filtering is necessary to take care of pileup of reads due to biases. I do that for alignment and SNP discovery, but think twice about it during de novo assembly. If no underlying genome is known, it is hard to tell whether the duplicated reads come from error or real sequence.

[zhidkov.ilia]

So when you assemble reads in to contigs, you will prefer that at least several reads will support the assembly. If you will have identical reads, you might obtain false contigs.

Ilia

[samanta]

It does not work that way for K-mer based assembler. Would you please explain your rationale? Why would one get false contigs?

[zhidkov.ilia]

Let me rephrase my last comment:
If duplicated reads don't contribute to downstream the de novo assembly pipe, it will be good idea to remove them.

Ilia