AfterQC: Automatic Filtering, Trimming, Error Removing and Quality Control for fastq

[[email protected]]

Hi luc,

When a sequencer is working, it may raise some bubbles from its liquid lanes. Bubbles can affect DNA synthesis reactions and wash processes. According to our investigation, Illumina NextSeq sequencers are much more probable to have bubbles. NextSeq sequencers are widely used in clinical environment, like cancer testing. For these quality critical applications, the reads in the bubble areas must be discarded due to they are usually wrong representations of the original DNA templates and may cause fake mutations. Since these reads usually have high quality scores, they cannot be filtered by regular quality filters. We found that there were more and longer polyG reads (a polyG read is a read has long sub-sequence of Guanine) in the bubble areas.

Based on this phenomenon, this debubble tool can visualise the bubbles, furthermore detect their contours, and then filter out the reads in bubbles.

[luc]

Hi Chen,

thanks, this looks interesting. What do you mean by this "* Detects and eliminates bubble artifact caused by sequencer due to fluid dynamics issues"?
How would you recognize such problems? Are the reads filtered out?

[[email protected]]

AfterQC

Code on github: https://github.com/OpenGene/AfterQC

Automatic Filtering, Trimming, Error Removing and Quality Control for fastq data

AfterQC can simply go through all fastq files in a folder and then output three folders: good, bad and QC folders, which contains good reads, bad reads and the QC results of each fastq file/pair.
Currently it supports processing data from HiSeq 2000/2500/3000/4000, X10, X5, Nextseq 500/550, MiniSeq...


Features:
AfterQC does following tasks automatically:

* Filters reads with too low quality, too short length or too many N
* Filters reads with abnormal PolyA/PolyT/PolyC/PolyG sequences
* Does per-base quality control and plots the figures
* Trims reads at front and tail, according to QC results
* For pair-end sequencing data, AfterQC automatically corrects low quality wrong bases in overlapped area of read1/read2
* Detects and eliminates bubble artifact caused by sequencer due to fluid dynamics issues
* Single molecule barcode sequencing support: if all reads have a single molecule barcode (see duplex sequencing), AfterQC shifts the barcodes from the reads to the fastq query names
* Support single-end sequencing or pair-end sequencing

Dependency:
AfterQC uses editdistance module, run following before using AfterQC:

pip install editdistance


Simple usage:
1, Prepare your fastq files in a folder
2, For single-end sequencing, the filenames in the folder should be *R1*
For pair-end sequencing, the filenames in the folder should be *R1* and *R2*


cd /path/to/fastq/folder
python path/to/AfterQC/after.py


Two folders will be automatically generated, a folder 'good' stores the good reads and a folder 'bad' stores the bad reads
AfterQC will print some statistical information after it is done, such how many good reads, how many bad reads, and how many reads are corrected.

Quality Control only
If you only want to get quality control statistics, run:
python after.py --qc_only