[rskr]

Benchmarking alignment algorithms was an interesting and exciting field in 2008 - in 2011, not so much.

Yep it was more important when you have 36mers to know how bad the alignments with them are. Now that we have 100mers, we can be a little more confident in our alignments.

[pmiguel]

Yep it was more important when you have 36mers to know how bad the alignments with them are. Now that we have 100mers, we can be a little more confident in our alignments.

Well, don't be coy. What benchmarking protocol would satisfy you? Please be specific.

--
Phillip

[rskr]

Well, don't be coy. What benchmarking protocol would satisfy you? Please be specific.

--
Phillip

BWA, comes with a test suite, that will generate randomized reads, and evaluate them.

Pretty easy, only reason not to do it is you know it isn't good.

wgsim does do some error modeling and biological zygosity, but it doesn't do a very sophisticated job of the error modeling such as quality and error decreasing at the ends of reads, so it is still somewhat optimistic. Also the scoring is a little odd it doesn't measure the exact SNPs, and indels, but only if the reads were in the right place +-5bp, and if a read maps wrong multiple times that counts against the error multiple. That could be improved, but it will give a pretty good idea of what the accuracy really is.

The best thing about bench marking? Its cheap!