gigaBayes (MarthLab) is running very slowly

[sdriscoll]

According to the documentation gigaBayes should run fairly quickly (18,000 30 bp reads per second) however on my system I'm seeing much slower performance (at least I'm interpreting it to be slow). With the --debug switch enabled I can watch the output and sometimes it spends 30 seconds or even several minutes at a single position resulting in total run time of more than 30 hours.

I'm using illumina sequence data, single end reads, mouse organism and I've run the data through MosaikAligner, MosaikSort, and finally MosaikAssembler to produce a GIG file for gigaBayes. We're only interested in investigating chromosome 7 so I only aligned my source data to that chromosome (truncated reference file). Alignment and the other two programs tend to complete in about 1 hour total but then gigaBayes runs for up to 2 days straight before completing and I've caught it using up to 15 GB of RAM. My full gigaBayes command looks like this:

gigaBayes --gig input.gig --gff output.gff --anchor --ploidy diploid --PSL 0.9 --debug

My system specs:
AMD Phenom 9950 Quad-core
16 GB RAM
Ubuntu 9.04 64 bit (2.6.28-15 kernel)

does anyone have any idea what's going on?

[sdriscoll]

Last night I tried running the above with a reduced region: 10,000,000 to 27,000,000. I ran that on 3 data files and each took about 1 hour to 1.5 hours to complete.

Today I tried something different. My revised command line looks like this:

gigaBayes --gig input.gig --gff output.gff --anchor --indel --ploidy diploid --sample "single" --debug

Those same lanes took around 3 minutes to complete each. So it must have been the --sample option. By default gigaBayes uses "multiple" meaning that your same came from multiple specimens (?). I'm not sure how that works but it doesn't apply to my data which all comes from one source. That still doesn't answer the performance question though since that's a default value and by default it should be running very quickly. For my purposes, I'm happy now. But I don't understand why there was such a massive speed difference. Again if anyone has any insight it'd be cool to hear it. Thanks.