reduce fastq redundancy for tophat?

[pengchy]

Hi, I wondering is it possible to reduce the fastq redundancy for the downstream analysis.

For RNAseq, because the cost decreased quickly, large number of data was produced for one sample, which may be not necessary, but always prefer by biologiest in the name for the low level expressed transcripts. However, in the same time, the highly expressed transcripts have sequenced many many times. So, there will be many many reads exactly same. But for the alignment tools, such as bowtie/tophat, all these reads will be processed and aligned to the big genome reference, although many many reads are exactly same.

The question is, is it possible to reduce the redundancy of the raw fastq reads while retain the copy information. When you do alignment, the multiple same reads only needed to align once. The quantity information will be integrated to measure the expression level. By doing so, the calculation will de decreased significantly.

[dpryan]

Have a look at some of the miRNA processing related programs (mIRexpress, for example). There, people will often collapse their fastq files into a set of unique reads and associated counts.

Having said that, tophat is usually fast enough (for me at least) that there's no benefit in doing something like that. If you have access to bigger hardware, give STAR a try...it is extremely fast.

[pengchy]

Hi dpryan,

Small RNA, like miRNA/piRNA, is very short, so it is feasible to reduce the redundancy simply using hash table. But the mRNA is not so easy.

Tophat indeed fast, but for huge genome size and huge rnaseq reads, it still slow. Say align 100Gb RNAseq reads to 6 Gb genome.

[dpryan]

Many of us use compute clusters, for just that reason.