Cufflinks runtime

[wilson90]

Dear all,

I have a pair-end RNA sample (80 millions reads in total), which has been aligned using tophat2, and run cufflinks on it. It has been 1 day but the program is still at the "Inspecting reads and determining fragment length distribution." phase. Is there anything wrong? Is there a case where Cufflinks went into infinite looping?

I use -g options. My Cufflinks is not of the latest version. I am running it on a clustered server.

Thank you.

Wilson

[wilson90]

Additional information on command:
cufflinks -g refGenes.gtf -p 35 -u -N --total-hits-norm -b genome.fa -q accepted_hits.bam

[gesdys]

I have the same problem, cufflink it's very very slow, and apparently it's not using the multithreading (option -p) properly. I don't know whether this is a bug or is the way they created.

[amarth]

Cufflinks takes a couples of hours with small samples, I've run 60 millions reads and it takes around 10 hours to complete, I have a Linux Workstation with these specs:

Dual-Core Intel Xeon Processor 5150, 4Cores, 2.66 GHz,, 16 GB DDR2 RAM and 1 TB HDD

If you're thinking if it's a problem, I recommend you to add the option [verbose] to make Cufflinks a little more informative of what's going on.

Samples Quality ALSO plays an important role to the time it takes to analize transcripts

Salutes

[gesdys]

did you check if the multi threading is working correctly?

[sdriscoll]

i'm not certain about this but i think there's a difference between multi-threadded programming (using multiple cores on a single workstation) and parallel programming (for a computing cluster). cufflinks is most certainly designed for multi-threadded execution on a single workstation but I don't think it's designed to fork it's processing across many different nodes on a cluster. I could be completely wrong but when running cufflinks on a cluster I think you're limited to the resources of a single node.

For 80 million reads, however, it shouldn't take very long at all to get through that first stage of determining the fragment length distribution and I'm not sure if that part even runs more than a single thread anyways. of course this depends on the CPU of the node you're running it on. imaging running 'samtools flagstat' on an alignment file - that's essentially what that step is with a bit of extra code. the multithreadded part is when it starts the assembly/quantification part. if you're using the '-b' option you're in for TWO trips through that process. that should also be the longest part of the process.

[gesdys]

I'm using a single workstation, and I checked: I used the verbose option, so I was able to see almost everything.
when it was calculating the number of reads for each transcript (this is a multithread part), it used more than one thread just for few transcript, then it started to use only one thread for all the other (that's why takes so long)...
I also check other programs such as bowtie and tophat and both of then are using multithreading perfectly...
so I really thing this is a bug of the last version of cufflinks...
hope they are gonna fix it, otherwise for huge dataset it will take forever...