Pipelining "Samtools view" : invalid header in output .bam files

[fabfab]

Hi,

I'm trying to filter properly paired and mapped reads from a BWA alignment using this command:

samtools view -hu -F4 bam_sorted.[COLOR="Red"]b[/COLOR]am | samtools view -hu -F256 - | samtools view -h -f3 - > test.bam

But the terminal tells me:

[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] EOF marker is absent. The input is probably truncated.

Despite these error messages, the output file (test.bam) is generated and seems to contain the reads I selected.
I red that I should not take care of this error messages. However, how I can be sure that there are no problems or that my output file is not truncated ?

But there's a more annoying consequence: I cannot handle further my test.bam file, because samtools says the formatting is incorrect !
example, if I try:

samtools view -c test.bam

I get again:

[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[main_samview] fail to read the header from "test.bam".

But if I open the file (using the -u option in the last pipe to avoid compression), I see that the header with @HD and @SQ is here ! So why do samtools not see it ?

Moreover, when I use "samtools view" to convert my .sam into .bam or to sort the previous .bam, everything works well (header is present and samtools can read the .bam file)... at least when I run these command in two steps not piping them.

I've read several threads on this issue, but I couldn't find a clear answer
(but I'm new to bioinformatics, so maybe I did'nt get it).
Please, could someone help me on this issue?

For info, I'm running samtools Version: 0.1.19-96b5f2294a.
Before, I ran BWA (Version: 0.7.5a-r405) for mapping

[maubp]

But the terminal tells me:

[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] EOF marker is absent. The input is probably truncated.

This warning can be a false alarm due to a bug in samtools v0.0.18 and v0.0.19 but which should be fixed in later releases - see https://github.com/samtools/samtools/issues/18

[maubp]

Hi,

I'm trying to filter properly paired and mapped reads from a BWA alignment using this command:

samtools view -hu -F4 bam_sorted.sam | ...

Are you really starting with a SAM input file? You need the -S (big S) to tell samtools view the input is SAM (it defaults to expecting BAM input).

[fabfab]

Hi,

In fact, I tried starting both with a .bam and .sam file (using the -S option in that case), but I ended up with the same error warning messages.
Excuse me, I made a mistake in copy-pasting my command lines, so my post is confusing. I edited my previous post to correct the code I use (in red)

Ok if this is a bug and I can neglect the error warnings.
But what about my main issue: I cannot use the output filtered .bam file with samtools because it is not recognized as a correctly formated BAM ?

Thanks for help,
Fabrice

[fabfab]

Hi again,

Ok, I got my mistake !
It might seem silly for most of you, but I write it down for beginners that could face the same issues...
In my previous pipe, the last ouput is not a compressed bam: I omitted the option -b:

before:

samtools view -hu -F4 bam_sorted.bam | samtools view -hu -F256 - | samtools view -h -f3 - > test.bam

modified code:

samtools view -hu -F4 bam_sorted.bam | samtools view -hu -F256 - | samtools view -[COLOR=Red][B]b[/B][/COLOR]h -f3 - > test.bam

I still have the error messages (that we don't care about)
But now, I can work with the output (test.bam) which is in a correct format recognized by samtools.