fastx quality trimmer and gzipped fastq

[balsampoplar]

Hello -

I am using fastq_quality_trimmer tool from the Fastx toolkit to do some quality control on my gzipped fastq file. However, I keep getting wrong file type error. Are the fastq files to be gunzipped prior to using them with this tool?

Edit 1: I figured out that I needed to use zcat to feed the data to this tool as follows:

$> zcat sequences_fastq.gz | fastq_quality_trimmer -t 30 -l 35 -z -o sequences_trimmed_fastq.gz

This results in the following error:
fastq_quality_trimmer: Invalid quality score value (char '#' ord 35 quality value -29) on line 4

I am not quite sure if I understand what's causing this.

[Brian Bushnell]

This is because fastx is obsolete and requires everything to use ASCII-64 qualities, but your data is modern and uses ASCII-33 qualities.

I suggest you use a different trimmer. For example, from the BBTools package:

reformat.sh in=sequences_fastq.gz out=trimmed.fq.gz qtrim=rl trimq=20 minlen=40

That will trim the left and right sides of reads to Q20 using the Phred method (superior to fastx's method), and discard resulting reads that are shorter than 40 after trimming (that parameter is optional).

I've attached a study I did recently on quality trimming using various trimmers. Note that each point on the graphs represents trimming with a different threshold, so the lower-left point for an algorithm is trimming to Q40, and the upper-right is trimming to Q0. 30 is generally way too high a trimming threshold for any purposes, by the way.

[SES]

This is because fastx is obsolete and requires everything to use ASCII-64 qualities.

This is not true, the FastX toolkit has had the '-Q 33' option for Sanger FASTQ data for maybe 4 years now. Though, there is the eternal question of why these things are not documented......but it's pretty common knowledge now. You will see this discussed quite a bit on seqanswers and biostars.

[Brian Bushnell]

This is not true, the FastX toolkit has had the '-Q 33' option for Sanger FASTQ data for maybe 4 years now. Though, there is the eternal question of why these things are not documented......but it's pretty common knowledge now. You will see this discussed quite a bit on seqanswers and biostars.

Oh, thanks for the correction; that would have made my testing easier. It should really be the default, otherwise anyone running modern data will get an immediate, mysterious crash.

[balsampoplar]

Thank you Brian and SES. As it turns out, I had to use both -Q and -t flags in order to successfully run the fastq_quality_trimmer. Using one of the other generated input formatting errors.

Unfortunately, the documentation of the program makes no mention of the -Q call. In fact if you access the help menu, -Q does not even show up. Also, what is the rationale behind having to use both flags? Don't they mean the same thing?