Quality filtering

[bharat_iyengar]

I am new to the field but from what I gathered is that it is essential to filter sequences with low quality (phred scores).

But what I generally observed is that I lose ~70-80% reads from some publicly available data when I apply a filter to remove reads with mean quality score < 20 (phred 33/illumina 1.8+) and individual nucleotide score < 10.

I am not sure whether it is a very stringent criterion.

When I asked one of the data submitters regarding the filtering criteria that they used he said that they didn't use any filters.

Can someone please tell me what is the right cut-off, which would both minimize data loss and preserve reliability.

[dpryan]

You might consider just quality trimming and keeping all reads above some minumum size (that's what I do). The ends of reads are often low quality, so trimming those off will probably help a lot here. There are a number of publicly available quality trimmers to do that (trimmomatic, etc.). I trim bases off with qual < 20 and just toss resulting reads less than 16 or so (the minimum will depend on what you're doing, your aligner, and if you have paired-end reads). Another possibility would be to also toss resulting reads with mean quality scores below 20 (after trimming, that number should be seriously reduced).

If you have paired-end reads, make sure whatever trimming program you use can handle that. There are a lot of posts on this forum from people with out of sync paired-end fastq files because they didn't do that.

[Apexy]

This is quite an old thread but you may consider novel perpectives when you engage with future datasets. If it is RNA-Seq you are dealing with, take a look at this paper http://journal.frontiersin.org/Journ...014.00017/full