alignment speed on workstation vs desktop

[a_mt]

Hi all,

Recently our lab purchased a workstation with 64 GB RAM, xeon processor (which has 64 cores) running on 64bit linux. I am connected to it through a desktop (linux 64bit) which has 4GB RAM, i5 processor (4 cores). I previously used to work on this desktop and now that I am connected to workstation, I thought of comparing the alignment speed on both machines.

Data I used is paired end, with each 1.3 GB.
I am using bowtie2 to align. (reference genome is yeast)

On workstation I had set threads (-p) argument to 58 cores, and on desktop I used 3 cores out of available 4.

Alignment results are same on both machines but the time taken by them is drastically different.

Desktop machine took 8:39 secs, whereas
Workstation took 18:18 secs !!!!!!!!

What might be the problem ??

I am sorry if my question is not relevent to the forum.

Thank you !

[dietmar13]

guess

perhaps time costs for parallelisation (splitting data and collecting results) are greater than time savings for mapping.

better use a few cores per data set and process different datasets simultaneously (e.g. using &).

only a guess...

[GenoMax]

Number of cores matters little if you can't efficiently feed them data (as you may have discovered).

You should experiment with ramping up the number of cores starting with a smaller value on the workstation and checking to see where you finally encounter a system-level bottleneck (specially in terms of I/O or memory bandwidth). You may end up finding that using 24 (or a smaller/bigger number for that matter) of cores gives you the fastest execution times.

You may be able to improve the I/O to some extent by using fast SSD drives (you did not say if you already have them) but other limits (memory bandwidth) would be properties of hardware you have in the workstation and can't be changed.

[xied75]

8:39 you mean 8 mins 39 secs?

I/O in this case shoudn't be the bottleneck, only a 1.3GB file, cost nothing to read.

Someone familiar with how bowtie2 does the multithreading?

If you like try with BWA ALN, report the number here.

Best,

dong

[a_mt]

@GenoMax :

You were right.. I used different number of cores this is what I found (attachment). At -p 30, it took about only 1 min 30 seconds, after that it went on increasing. I think this is where bowtie2 encountered system-level bottleneck.. May be I will stick with this number for a while.. Thank you all for the suggestions..