BWA multithreading

[kga1978]

Hi all,

I have BWA installed on a Mac OS 10.6 installation with 8 cores (64 bit machine). I have tried using '-t 8' or '-t 4' to use multiple cores, but it doesn't appear to be working? I have built the program from source, but don't seem to be able to find anything I can change in the Makefile.

Any help would be much appreciated.

Thanks

[kga1978]

Any thoughts?

[nilshomer]

You do not give sufficient detail (input data, command line) to debug the problem. Thanks!

[kga1978]

Hey Nils,

Sorry, didn't realize that. Everything has been run pretty standard - here's an example:

bwa aln -q 5 -l 32 -k 2 -o 1 -t 4 database.fasta reads.fastq > aligned.sai

As I said - the '-t 4' parameter is the problem as bwa doesn't appear to be able to use multiple cores on my install - and in fact, when I use -t 4, it will slow down the program compared to if that option isn't used.

I have tried rebuilding bwa from source (two different versions - - either using 'make' or the 'sh autogen.sh' method - neither works).

At the end of the day, I guess bwa isn't very suited for my purposes (my data is quite divergent), but the Broad is using bwa as their pipeline, so I will have to try and work with it for now - multi threading would help ;-).

[adaptivegenome]

Provide more detail and maybe we can help. We run BWA on a 64 core machine and it runs great. It scales nearly perfectly (64 cores = 64X speed up).

[Richard Finney]

make fails for bwa?
What's the error message?

Make sure you're compiling with -DHAVE_PTHREAD When you type make you should see this "string" as parameter to gcc. If you don't have the pthread library, then it won't link.

[kga1978]

make appears to be working and I don't see any errors popping up. On a fresh install, this is what is written to the terminal window:

$make
make[1]: Nothing to be done for `lib'.
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwt_gen.c -o bwt_gen.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD QSufSort.c -o QSufSort.o
ar -cru libbwtgen.a bwt_gen.o QSufSort.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD utils.c -o utils.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwt.c -o bwt.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwtio.c -o bwtio.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwtaln.c -o bwtaln.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwtgap.c -o bwtgap.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD is.c -o is.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bntseq.c -o bntseq.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwtmisc.c -o bwtmisc.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwtindex.c -o bwtindex.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD stdaln.c -o stdaln.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD simple_dp.c -o simple_dp.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwaseqio.c -o bwaseqio.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwase.c -o bwase.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwape.c -o bwape.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD kstring.c -o kstring.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD cs2nt.c -o cs2nt.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwtsw2_core.c -o bwtsw2_core.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwtsw2_main.c -o bwtsw2_main.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwtsw2_aux.c -o bwtsw2_aux.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwt_lite.c -o bwt_lite.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwtsw2_chain.c -o bwtsw2_chain.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bamlite.c -o bamlite.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD main.c -o main.o
gcc -g -Wall -O2 -DHAVE_PTHREAD utils.o bwt.o bwtio.o bwtaln.o bwtgap.o is.o bntseq.o bwtmisc.o bwtindex.o stdaln.o simple_dp.o bwaseqio.o bwase.o bwape.o kstring.o cs2nt.o bwtsw2_core.o bwtsw2_main.o bwtsw2_aux.o bwt_lite.o bwtsw2_chain.o bamlite.o main.o -o bwa -lm -lz -lpthread -Lbwt_gen -lbwtgen

My computer is a Mac Pro running 10.6.8 and as far as I can tell pthread should be there. I just use the standard Makefile with no modifications - everything appears to be in order - here's a copy of the file:

CC= gcc
CXX= g++
CFLAGS= -g -Wall -O2
CXXFLAGS= $(CFLAGS)
DFLAGS= -DHAVE_PTHREAD #-D_FILE_OFFSET_BITS=64
OBJS= utils.o bwt.o bwtio.o bwtaln.o bwtgap.o is.o \
bntseq.o bwtmisc.o bwtindex.o stdaln.o simple_dp.o \
bwaseqio.o bwase.o bwape.o kstring.o cs2nt.o \
bwtsw2_core.o bwtsw2_main.o bwtsw2_aux.o bwt_lite.o \
bwtsw2_chain.o bamlite.o
PROG= bwa
INCLUDES=
LIBS= -lm -lz -lpthread -Lbwt_gen -lbwtgen
SUBDIRS= . bwt_gen

.SUFFIXES:.c .o .cc

.c.o:
$(CC) -c $(CFLAGS) $(DFLAGS) $(INCLUDES) $< -o $@
.cc.o:
$(CXX) -c $(CXXFLAGS) $(DFLAGS) $(INCLUDES) $< -o $@

all:$(PROG)

lib-recur all-recur clean-recur cleanlocal-recur install-recur:
@target=`echo $@ | sed s/-recur//`; \
wdir=`pwd`; \
list='$(SUBDIRS)'; for subdir in $$list; do \
cd $$subdir; \
$(MAKE) CC="$(CC)" CXX="$(CXX)" DFLAGS="$(DFLAGS)" CFLAGS="$(CFLAGS)" \
INCLUDES="$(INCLUDES)" $$target || exit 1; \
cd $$wdir; \
done;

lib:

bwa:lib-recur $(OBJS) main.o
$(CC) $(CFLAGS) $(DFLAGS) $(OBJS) main.o -o $@ $(LIBS)

bwt.o:bwt.h
bwtio.o:bwt.h
bwtaln.o:bwt.h bwtaln.h kseq.h
bwt1away.o:bwt.h bwtaln.h
bwt2fmv.o:bwt.h
bntseq.o:bntseq.h
bwtgap.o:bwtgap.h bwtaln.h bwt.h

bwtsw2_core.o:bwtsw2.h bwt.h bwt_lite.h stdaln.h
bwtsw2_aux.o:bwtsw2.h bwt.h bwt_lite.h stdaln.h
bwtsw2_main.o:bwtsw2.h

cleanlocal:
rm -f gmon.out *.o a.out $(PROG) *~ *.a

clean:cleanlocal-recur

[kga1978]

Just in case anybody else ran into this problem. The lack of multi-threading was caused by the size of the reference file I was using - my genome is only 4kb, so I guess bwa couldn't throw all the data at 8 cores - this just slowed down the program, rather than speeding it up. Using hg19 as a reference, everything is working great.