SOAP3-dp - the potential bwa killer?

[ymc]

http://developer.download.nvidia.com/GTC/PDF/GTC2012/PresentationPDF/S0109-GTC2012-Soap3-GPU-Alignment.pdf

Supposedly can do gapped alignment. Anyone tried?

Speed-wise, it might be able to kill bwa when GK110 comes out.

[adaptivegenome]

Would be nice to see an ROC and a speed test before declaring BWA dead...

[xied75]

Diamonds are forever... (with Bond tune)

Just look how all these mappers started, all just being descendents of Eland, and why? Because they have a machine thus they have the C source code.

And everything using BWT, when was the BWT paper published? 1994. What about Suffix Tree? 1976. Are you trying to say Bioinformatics is cutting edge?

From Blast to Eland was a jump; from Eland to BWT was a jump; from BWT to... well, using GPU, I also see people using FPGA, so maybe just half a jump. I guess a big problem to solve is how to use Intel ManyCore, with everyday C lib, so that your code could run on a laptop with 2x 100 cores CPU there, other than wrap your data into a structure and off load to the GPU to speed up your for loop only. Just my 2 pence.

[Richard Finney]

Show me the benchmarks ... they are there ... on page 16.



I'm guessing the catch is BWA is single threaded.
... or maybe BWA is just a single process; BWA is easily parallelize-able for multi-cores or multi-machine via splitting up the data to be input to BWA into many chunks.

[adaptivegenome]

Show me the benchmarks.

Speed and ROC please.

[ymc]

Show me the benchmarks ... they are there ... on page 16.



I'm guessing the catch is BWA is single threaded.
... or maybe BWA is just a single process; BWA is easily parallelize-able for multi-cores or multi-machine via splitting up the data to be input to BWA into many chunks.

I think it will be unlikely for BWA to run faster than soap3-dp even on the lastest 16-core dual Xeon setup. Anything beyond a dual Xeon setup will be too expensive (in terms of money, space and power) to be compared to a single GPU setup.

Plus, when Tesla K20 or GTX 780 comes out, the gap between computational power of CPU and GPU will be huge.

[adaptivegenome]

I think it will be unlikely for BWA to run faster than soap3-dp even on the lastest 16-core dual Xeon setup. Anything beyond a dual Xeon setup will be too expensive (in terms of money, space and power) to be compared to a single GPU setup.

Plus, when Tesla K20 or GTX 780 comes out, the gap between computational power of CPU and GPU will be huge.

And what about accuracy? There are lots of ways to speed up mapping, most involve sacrifices to accuracy. And I do not follow your logic regarding the inability of BWA to run faster than SOAP3. Would be nice to see a benchmark that is apples to apples. Run BWA as Richard suggested or just run SOAP3 with 1 CPU core + GPU so we can truly see the contribution of the GPU.

[ymc]

And what about accuracy? There are lots of ways to speed up mapping, most involve sacrifices to accuracy. And I do not follow your logic regarding the inability of BWA to run faster than SOAP3. Would be nice to see a benchmark that is apples to apples. Run BWA as Richard suggested or just run SOAP3 with 1 CPU core + GPU so we can truly see the contribution of the GPU.

Well, I don't know about SOAP3-dp's accuracy. Throughout my posts, I am only saying it will be substantially faster than bwa when the new Kelpers come out.

To make the comparison fair, I think it should base it on price and form factor. So I would think a comparison of Tesla with dual Xeon setup would work for people who are building clusters. For home users, GPUs will always have better bang for the bucks.

There is a SOAP3 paper listed at the end of the pdf. You can check that out if it mentions anything about accuracy. But this paper is not about SOAP3-dp though.

BTW, I am not related to this SOAP3 effort. I have never used it either. I am posting this because recently I am into GPU computing and find this on nvidia web site.

[xied75]

I think it will be unlikely for BWA to run faster than soap3-dp even on the lastest 16-core dual Xeon setup. Anything beyond a dual Xeon setup will be too expensive (in terms of money, space and power) to be compared to a single GPU setup.

Plus, when Tesla K20 or GTX 780 comes out, the gap between computational power of CPU and GPU will be huge.

I very much doubt it. With my Windows BWA on Windows x64 using 32 threads (-t 32, 16 real cores, with HyperThreading on), I can generate SAI file on read 1 in 10 mins, read 2 in 11 mins, each with 38.5 million 83 bp reads. In theory it should be faster with 32 real cores.

Price, space, and power wise, I can get a DELL 1U R715 with 2x AMD 16 cores for 3000 pounds, with Redundant PSU max to 750W (normal run < 300W). Now how much is your Tesla costs? How much is your machine to host the Tesla costs? How much power is that drawing?

By no means I say using GPU is bad, but in IT world (that's another world from Bioinfo), there is a concept called TCO - Total Cost of Ownership. This includes above factors and also the cost to maintain/further develop the code community wise. GPU code could be 10x-100x harder to understand and debug, right?!

GPU core count increases very fast, but CPU is catching up, Intel Manycore could do 100 now, there is nothing stop them to have more, only problem is people don't know how to use it, except some server software like SQL Server etc.

You may want to check this thread as well, there is a mapper called SNAP, it already can run 10x faster than BWA if you give it >40GB memory. http://seqanswers.com/forums/showthread.php?t=20172

SOAP3 on Nvidia web should have been there for quite a while, I remember seeing it last year.

[ymc]

I very much doubt it. With my Windows BWA on Windows x64 using 32 threads (-t 32, 16 real cores, with HyperThreading on), I can generate SAI file on read 1 in 10 mins, read 2 in 11 mins, each with 38.5 million 83 bp reads. In theory it should be faster with 32 real cores.

Price, space, and power wise, I can get a DELL 1U R715 with 2x AMD 16 cores for 3000 pounds, with Redundant PSU max to 750W (normal run < 300W). Now how much is your Tesla costs? How much is your machine to host the Tesla costs? How much power is that drawing?

As a matter of fact, you can see that GPUs are now dominating the latest supercomputer clusters. Also, if you check the GFLOPS per watt for the latest generation of Nvidia GPUs, you will understand why.

By no means I say using GPU is bad, but in IT world (that's another world from Bioinfo), there is a concept called TCO - Total Cost of Ownership. This includes above factors and also the cost to maintain/further develop the code community wise. GPU code could be 10x-100x harder to understand and debug, right?!

Saying "GPU code 10x-100x harder to understand" means you never programmed any GPU code. Due to limited signaling capabilities, GPU device code can't get very complicated compared to their CPU bound threaded code.

GPU core count increases very fast, but CPU is catching up, Intel Manycore could do 100 now, there is nothing stop them to have more, only problem is people don't know how to use it, except some server software like SQL Server etc.

There is a trend to make a CPU looks more like a GPU. But if the foreseeable future, they are not comparable.

SOAP3 on Nvidia web should have been there for quite a while, I remember seeing it last year.

I know that SOAP3 has been around for ages but it couldn't do gaps. This thread is about SOAP3-dp.

[arvid]

Well, I don't know about SOAP3-dp's accuracy. Throughout my posts, I am only saying it will be substantially faster than bwa when the new Kelpers come out.

While I do care about performance of the software I use, it won't help me at all if my alignment is finished in 5 minutes instead of an hour, if it isn't as accurate and sensitive.

So I don't quite get the hype about the new fast GPUs and SOAP3-dp - show me the ROC curves and I'll be interested and might even consider GPU boards for our servers (if it is reliable and won't break the warranty on the machines)...

There are a couple of BWA CUDA implementations - a quick search found BWA-CUDA and BarraCUDA - how do these compare to SOAP3-dp and the standard BWA in terms of accuracy and sensitivity - any shortcuts taken?

[ymc]

While I do care about performance of the software I use, it won't help me at all if my alignment is finished in 5 minutes instead of an hour, if it isn't as accurate and sensitive.

So I don't quite get the hype about the new fast GPUs and SOAP3-dp - show me the ROC curves and I'll be interested and might even consider GPU boards for our servers (if it is reliable and won't break the warranty on the machines)...

There are a couple of BWA CUDA implementations - a quick search found BWA-CUDA and BarraCUDA - how do these compare to SOAP3-dp and the standard BWA in terms of accuracy and sensitivity - any shortcuts taken?

I want to know the ROC, too. Unfortunately, while the SOAP3-dp software is out, no papers about it are out yet.

[aeonsim]

Possibly interesting but like the others until one can see ROCs comparing it to the other aligners to see where the trade offs are it's just another aligner. Percentage of Reads aligned is a pretty meaningless statistic if there's no measure of alignment accuracy.

See http://lh3lh3.users.sourceforge.net/alnROC.shtml for an example of what's needed.

Secondly there is a significant cost involved in acquiring suitable hardware to run this SOAP3-dp, specialized hardware that's going to be of little use for any other stage of a bioinformatics pipeline (unless someone has come up with a GPU based Variant caller).

If this does end up being of similar sensitivity and specificity to BWA or better, and for groups that have alignment as there limiting stage with regards to performance then this may well be very useful, otherwise you'll likely do better investing in additional general compute capacity that boosts performance at all stages of the pipeline. After all Mapping is only a part of a full bioinformatics pipeline for NGS (mapping, pcr duplicate detection, Realignment for indels, Variant calling, Annotation & filtering).

Any way I'll keep an interested eye out for the paper or someone doing some decent analysis of it.

[ymc]

Possibly interesting but like the others until one can see ROCs comparing it to the other aligners to see where the trade offs are it's just another aligner. Percentage of Reads aligned is a pretty meaningless statistic if there's no measure of alignment accuracy.

See http://lh3lh3.users.sourceforge.net/alnROC.shtml for an example of what's needed.

Secondly there is a significant cost involved in acquiring suitable hardware to run this SOAP3-dp, specialized hardware that's going to be of little use for any other stage of a bioinformatics pipeline (unless someone has come up with a GPU based Variant caller).

If this does end up being of similar sensitivity and specificity to BWA or better, and for groups that have alignment as there limiting stage with regards to performance then this may well be very useful, otherwise you'll likely do better investing in additional general compute capacity that boosts performance at all stages of the pipeline. After all Mapping is only a part of a full bioinformatics pipeline for NGS (mapping, pcr duplicate detection, Realignment for indels, Variant calling, Annotation & filtering).

Any way I'll keep an interested eye out for the paper or someone doing some decent analysis of it.

There is also one GPU variant caller called GSNP.

http://dl.acm.org/citation.cfm?id=2066903

Even if these program's ROC is not as high, I think there will still be cases where its speed up is useful. After all, BWA itself is not running Smith-Waterman either.

[adaptivegenome]

There is also one GPU variant caller called GSNP.

http://dl.acm.org/citation.cfm?id=2066903

Even if these program's ROC is not as high, I think there will still be cases where its speed up is useful. After all, BWA itself is not running Smith-Waterman either.

Accuracy is more important than speed. But again after all these messages none of us are any closer to knowing the accuracy or speed of this code.