bedtools bamtofastq taking too long

[ty23991]

Hi
I am running the bedtools bamtofastq for converting bam to splitted fastq (read1 and read2) files for paired end reads with an expected wall time of 24 hours and memory of 4000mb.

The input bam file is about 140 GB whereas the output file size has reached to merely 0.3 GB. So it seems that the tools is running too slow for some reason. The following is command was used. The input file has been sorted by samtools sort by names.

bedtools bamtofastq -i X_Sorted.bam -fq X.R1.fq -fq2 X.R2.fq

Any suggestions appreciated.

[GenoMax]

Perhaps you have hit memory/wall time limit for this job?

An alternative may be reformat.sh from BBMap suite. Brian's code tends to be very efficient so you may want to give it a try. Make sure samtools is in your path for something like following to work.

$ reformat.sh in=file.bam out=file.fq.gz

Are you sure that both reads have been kept in the bam file? If not you will have a problem of singletons and you will need to separate them before de-interleaving the fastq file. Start with this:

$ reformat.sh in=file.fq.gz verifypairing
$ reformat.sh in=file.fq.gz out1=R1.fq out2=R2.fq

[ty23991]

Thanks so much. It turned out that the output directory was located in a different file system in a cluster and that somehow causes bedtools output to be written at extremely slow rate. The problem solves if i use the output directory location within the same file system in the cluster. I will post more details in case this happens only with bedtools or is a general I/O issue.

Regardless- I tried reformat.sh and based on output rate, it turns out to be relatively efficient. Thanks so much for the clue.

[muxingu]

Use the "bamtofastq" in Biobambam instead. It runs way faster and the code is highly optimized. Bedtools is written by amateurs. That's why it's slow.