An important bug in BLAST+ 2.2.26+

[benjwoodcroft]

Hi everyone,

TL;DR: Always use the -max_target_seqs flag when using BLAST unless you are using default verbose output, otherwise you might not get all the hits you should.


Example:

$ blastn -perc_identity 97 -query my_query.fa -db nt -out result.txt
$ blastn -perc_identity 97 -query my_query.fa -db nt -outfmt 6 -out result.csv
$ grep '>' result.txt |wc -l
186
$ wc -l result.csv
143 result.csv

So you get different numbers of results using default textual output and csv format. I was able to 'rescue' it by adding the -max_target_seqs flag:

$ blastn -perc_identity 97 -max_target_seqs 500 -query my_query.fa -db nt -outfmt 6 -out result.csv2
$ wc -l result.csv2
186 result.csv2

I confirmed this on the blast-help email helpdesk at NCBI. Their response:

For output formats >4, -max_target_seqs should be explicitly set. In the next release, 2.2.27, you should get a commandline message to that effect

This is true whether or not you use the -perc_identity flag as I did.

Hope that helps someone, and does affect too many different pieces of software and science..
ben

--
Tyson Laboratory, Australian Centre for Ecogenomics

http://ecogenomic.org

[SOLiDance]

Really? It's pretty serious one, Thanks! I'll keep it in mind

[maubp]

Wow - that sounds rather nasty. But what shocks me is the reply from the NCBI - are they really saying that in 2.2.27+ they won't fix it, they'll just add a warning!?

[benjwoodcroft]

are they really saying that in 2.2.27+ they won't fix it, they'll just add a warning!?

Potentially, we'll see. It might instead mean that it will become a required parameter, rather than just a warning. That doesn't seem perfect either though because software that runs blast under the hood will now start failing.

[kmcarr]

I have to say I don't really see this as a bug at all. Different reporting modes naturally take different parameter settings. Defaults for those parameters are set at some value reasonable and appropriate for the reporting mode chosen. It is up to the user to understand how the parameters chosen will effect the output. I will grant there may be some lack of clarity in the current documentation of the --max-target-sequences setting but the program is working as designed so you can't call that a "bug" and it doesn't need fixing.

[maubp]

You say undocumented 'feature', I say bug. And a nasty one as it causes silent data loss.

[BEpier]

Hi,

Do you have any idea on how the not reported hits behave? Do they share some common pattern or anything to be deleted or is it just something random?

Thanks for the info!