Tophat Run time

[bassu]

Dear all,

I am currently running tophat to align human genome reference(hg19 bowtie index) to illumina pair-end data of size 17Gb each(17*2). Its almost 24 houurs and its still running. I wonder how much more time it would take to finish the process? . Concurrently i am also running Maq for 75bp single read of size 15GB with a reference size of 30mb(binary human genome file).

My system configuration is 64Gb Ram with 8 core processor(Which i feel is one among best configuration available in industry. Do i need to update my system configuration for NGS data analysis?If so please provide me the config).

And i also like to know how much processing time it would take if i run the Tophat and Maq separately?

Hopping for a speedy reply asap.

Thanks

[Rao]

try with bowtie...
is it RNA-seq data?

[lmf_bill]

Using tophat, especial for paired-end reads, it will take long time. In my personal point, your configuration is enough.
BTW, tophat will produce huge tmp file.

[john_mu]

Did you run TopHat with multiple threads?

If you are running it with only one thread 17Gb of reads will take several days to run...

What is your read length? Long reads also take much longer than short ones (for the same amount of data).

Running TopHat and Maq at the same time should not cause much problems (Unless you ran TopHat with 8 threads)

Regarding your system configuration, 64Gb should be plenty of RAM for your amount of data.

[bassu]

Thanks all,
@Rao: yes i am using RNA-seq data.

@mf_bill: thanks for your valuable comment, I was wondering whether my system configuration was right ? Even though BWT, tophat will produce huge tmp files.. it will get deleted automatically right?

@john_mu: Thanks john, currently i'm running my reads in a single thread. and my read length is 50bp.

[DineshCyanam]

@Bassu: So how long did it take for you to finally finish the run? Were u able to reduce the run time in any way? I am having the same problem here...

[mrawlins]

Running with n-1 threads on an n-core machine (e.g. 7 threads on a machine with 8 cores) should speed things up. Bowtie has a --shmem option for using shared memory for all threads, so that shouldn't increase the memory footprint by much to use that many threads. I've observed roughly a linear speed up with the number of cores dedicated to bowtie; I suspect similar results for tophat.
I will sometimes run with as many threads as cores, but only if I don't intend to use the computer for anything else while the program runs (i.e. a compute node on our analysis cluster).

[crazyhottommy]

I was running Tophat to map a 24G single end RNA-seq fastq to hg19 with the gtf from GENECODE
I run it in a cluster with 1 node, 8 processors, ram=3gb
it took me 45hrs to finish.....

Any way to speed it up? as a regular user of the cluster, the above setting is the max resource I can have.

[dpryan]

I was running Tophat to map a 24G single end RNA-seq fastq to hg19 with the gtf from GENECODE
I run it in a cluster with 1 node, 8 processors, ram=3gb
it took me 45hrs to finish.....

Any way to speed it up? as a regular user of the cluster, the above setting is the max resource I can have.

Split the fastq file and use multiple nodes. If you had more RAM, you could run STAR.

[shi]

You may try Subread, which is >10 times faster.

[crazyhottommy]

You may try Subread, which is >10 times faster.

I will give it a shot. Thanks

[adamyao]

20.8G bases (104M reads, Pair End) 1 node (16 cores) AMD 2.4GHz 96G ram
- 28 hours
Tophat 2 runs best with 16 cores ( single node) according to our tests otherwise it takes longer.
STAR runs much faster ( less than 40 minutes) but needs a lot more memory (64 cores 128G ram).