Data preprocessing

[Brian Bushnell]

The problem is not so much the fraction of data that is discarded, but rather, the bias - Illumina read quality is affected by sequence content, so a high quality-trimming or quality-filtering thresholds can disparately impact certain genomic regions. This is particularly important for quantitative analyses. And regardless of bias, longer reads give more accurate mapping. The confidence of a 250bp alignment, and the ability to place it correctly despite inexact repeats in the genome, is much higher compared to a 150bp read, even if the last 100bp of the 250bp read are only Q17 and thus would be expected to contain 2 mismatches. For variant-calling, q-trimming can be done AFTER alignment to allow the most accurate mapping possible.

[deep639]

I disagree with the part about quality-trimming. There is at least one published study indicating trimming to high levels like Q20 is generally detrimental to alignment, which agrees with my observations. However, it depends on the aligner. For something like bowtie1 which can only tolerate 3 mismatches, long reads >100bp might indeed need very aggressive quality-trimming... but then, you shouldn't use bowtie1 with long reads.

I generally suggest the range of Q5-Q12 for quality-trimming using HiSeq/MiSeq Illumina data that has the full range of quality scores. Illumina is moving toward binned and inaccurate Q-scores on its latest platforms, though, so the utility of quality-trimming is going to be reduced.

I think trim_galore by default sets it quality trimming to 20, I have never tried to change the setting, usually a small percentage of reads are thrown out because of quality and adapter trimming.

[Brian Bushnell]

Its always good to trim the adapters and do quality trimming before running alignment. Quality trimming is usually done at q20 level. Programs like trim_galore can auto detect the adapter sequence that needs to be trimmed based on the input reads.

I disagree with the part about quality-trimming. There is at least one published study indicating trimming to high levels like Q20 is generally detrimental to alignment, which agrees with my observations. However, it depends on the aligner. For something like bowtie1 which can only tolerate 3 mismatches, long reads >100bp might indeed need very aggressive quality-trimming... but then, you shouldn't use bowtie1 with long reads.

I generally suggest the range of Q5-Q12 for quality-trimming using HiSeq/MiSeq Illumina data that has the full range of quality scores. Illumina is moving toward binned and inaccurate Q-scores on its latest platforms, though, so the utility of quality-trimming is going to be reduced.

[deep639]

Its always good to trim the adapters and do quality trimming before running alignment. Quality trimming is usually done at q20 level. Programs like trim_galore can auto detect the adapter sequence that needs to be trimmed based on the input reads.

[Dario1984]

It's always best to do adapter trimming, even if the aligner you use later can do soft-clipping, such as STAR. It would work moderately faster if it didn't have to attempt to map the adapter sequences to the genome and then soft-clip the sequences when no alignment was found. You can have a look at the overall quality profile of the reads in a sample using a tool like FastQC. It's worth applying quality filtering, even if very few reads in the dataset are removed by it. There could be a rare circumstance where it makes an important difference.

[GenoMax]

In general you can do without explicit quality filtering, unless you know that you data contains a significant portion of low quality reads.

If you are aligning to a known reference you can get away with using data that may be as low as Q10. If you are doing de novo work then that would likely be the only case where quality score based filtering is warranted. You would want to filter out data Q25 or lower for that application.

[Katerina Cvikova]

Hi,
do you use a some rule for trimming/removing or saving reads with the worst quality for DNA and RNA assembly/alignment. How do you decide to trimm/remove/save? According the type of analysis, coverage, base quality.... ???? Thanks a lot.