blast+ on a grid

[yaximik]

Hi,

After reading a lot manual pages I am still uncertain how to properly schedule a multithreaded blast run with SGE, so please help. I wanted to run blastx with option of 16 threads, but I cannot find how to request resources for these 16 threads in a qsub script. The manual gives openmpi examples, followed by either mpiexec or mpirun, but blast+ is not said to be compiled for the openmpi environment, is it? Another available option would be '-pe smp 16', but is not that requesting one node with 16 cores, which may not exist? Other described arguments to qsub for grids elsewhere mention slots=n, cores=n, low* n, high n, threaded n, orte n, orte_fillup - all these return error on our grid.
Another option would be running blasx with GNU parallel, but again it is not clear how to request number of threads/cores/CPUs in such case.

[mike.t]

On your head node type 'man qsub' to get the man page you need. Its been a while since I've used SGE. I think you need a line like this

#$ -pe 16

in your submit script.

I think you'll get much better performance by splitting your query sequences into 16 separate fasta files and submitting an array job.

[maubp]

If you have 16-core machines on the cluster, use -pe smp 16, and call BLAST+ with 16 threads. If you only have 4-core machines on the cluster, use -pe smp 4, and call BLAST+ with 4 threads.

As Mike points out, when you query file is made up of lots of sequences, splitting this into separate FASTA files and running separate BLAST processes on different cluster nodes makes sense (you can then combine their output files).

BLAST+ itself has no built in capabilities like this, something like MPI-BLAST does but is based on the legacy C BLAST suite and quite old now.