Try to enhance the efficiency of SAMtools merge option

[Tonnny97]

Hi Guys,

Now I have around 100 sorted .bam files (~150MB each one) generated from TopHat.
Then I need to merge all these .bam files by SAMtools.

I try to enhance the efficiency on Hadoop cloud platform.
There are two choices:
1) Merge all files in one SAMtools merge command (single machine)
2) Merge each pair of two files in the first level, and then merge each pair of two files (from 1st level) in the second level, and ... (the concept similar to merge sort)

My question is:
Whether the second choice can run more faster?
Does anyone try to further enhance the merge efficiency of SAMtools?
Thanks.

[Heisman]

Another thought would be to try Picard. I do not know if it is faster or not but it may be. It will handle headers more appropriately: http://picard.sourceforge.net/comman...#MergeSamFiles

[lh3]

Samtools is faster than Picard for most operations. It also has parallel compression which picard lacks if I am correct.

I am not sure why you want to merge 100 RNA-seq BAMs. If you have to do that, the two approaches won't differ much. The bottleneck is writing the final BAM. You may try novosort from novocraft which takes better advantage of multi-core, but it is not free. You may also consider to use option "-1". Your output will be ~10% larger but the compression speed will be several times faster.

[Tonnny97]

Thanks for the suggestion!!