BFAST wish list

[nilshomer]

First of all, I am sorry for your frustration. You are welcome to contribute to the code since I think you have some good suggestions (it is Open Source). Make sure you are using the latest version.

Hello, I'm running a sensitivity/specificity study between aligners for my master thesis and I've noticed that BFAST is really slow when reading each one of the 14GB indexes.

I assume you are not talking about index creation. The indexes are a compressed. Feel free to remove that compression (gzip) and they will read in to memory faster. We have a fast file system (lustre) and it takes a while to load the indexes (a few minutes per), but given you batch 25M or more reads, it is not that bad. Again feel free to play with the code.

I have a 24 Core Xeon and the problem is that at some points it only uses 1 CPU instead of the 16 that I specified, why is this?

Not everything is parallelizable or scales linearly (see Amdahl's Law). For example, reading in an index, merging the results (you can do parallel merge sort), or outputting in sorted order.

After using blat for so long I'm so happy somebody patched it up. Also what is the deal with reporting negative aligments and not updating the flag field in SAM, can I get a 4 there?

It reports all alignments and lets the user filter the data. If you report too few alignments, then a new alignment must be performed to recover them. The SAM flag has a "4" since it is binary encoded (see the SAM spec). Try "samtools view -X <in.bam>" to see the flag in a string representation. If you see a "4" when you don't expect it, submit a bug, we are quite responsive.

It would also be nice if when you set threads to something other than 2n it would tell you its invalid before you have to wait the indexes to load would have saved me a lot of time.

What version are you using? The threads do not have to be a multiple of two when aligning. Are you talking about the one-time indexing step?

PS:
The only thing missing from BFAST is CUDA support, anybody working on this?

Sounds like a good master's thesis project

How nice would it be if you could do 512 threads at the same time, I'm sure you'll need it with Hi-Seq.. Cloud computing doesn't work if you have to upload 200GB of sequence data every day, a $1000 GPU makes much more sense. You are waisting time trying to run a simple text matching algorithm in a CPU, GPU's sounds better..

That last sentence sounds too simplistic. The goal of BFAST was to first meet the sensitivity requirement, THEN be as fast as possible. (Warn: generalization) biologists want the right answer (eventually), not the wrong answer (quickly).

[aleferna]

Hello, I'm running a sensitivity/specificity study between aligners for my master thesis and I've noticed that BFAST is really slow when reading each one of the 14GB indexes. I have a 24 Core Xeon and the problem is that at some points it only uses 1 CPU instead of the 16 that I specified, why is this? After using blat for so long I'm so happy somebody patched it up. Also what is the deal with reporting negative aligments and not updating the flag field in SAM, can I get a 4 there? It would also be nice if when you set threads to something other than 2n it would tell you its invalid before you have to wait the indexes to load would have saved me a lot of time.

PS:
The only thing missing from BFAST is CUDA support, anybody working on this? How nice would it be if you could do 512 threads at the same time, I'm sure you'll need it with Hi-Seq.. Cloud computing doesn't work if you have to upload 200GB of sequence data every day, a $1000 GPU makes much more sense. You are waisting time trying to run a simple text matching algorithm in a CPU, GPU's sounds better..