Samtools....SAM to BAM...warning or error!! Can I ignore?

[Siva]

Hello All

I am trying to convert a Tophat SAM file to BAM and I get the warning message "The tag [ID] required for [PG] not present.". I have included it below. However then [sam_header_read2] sequences are loaded and the program runs successfully.

./../../samtools-0.1.7a/samtools import ../../../ZmB73_AGPv1_genome.fasta.fai accepted_hits.sam accepted_hits_complete.bam
The tag [ID] required for [PG] not present.
[sam_header_read2] 11 sequences loaded

.

fyi...the first ten lines of the sam file:

@HD	VN:1.0	SO:sorted
@PG	TopHat	VN:1.0.13	CL:/share/apps/tophat-1.0.13/bin/tophat -o ./tophat_out_s8 -p 4 --solexa1.3-quals ./maize_genome_bwtind/ZmB73_AGPv1_genome_ind s_8_sequence.txt
HWI-EAS313:8:102:1328:1633#0	16	chr1	421	3	42M	*	GATTTCCAGTACAGTCCTCGCTATTGCTGTGAAAAGTTGGCC	@A??=A?B@@?BB@ABA>BA@BBB@:BABBBBBBB@BCBABB	NM:i:0
HWI-EAS313:8:91:606:9#0	0	chr1	449	255	42M	*	0	GTGAAAAGTTGGCCTCATATTCTTGGCTCCTCTTCAAAAAGA	B@B@BB@B?@A@ABB@BBAB?@>=>??;<<=:?:07:4?8+<	NM:i:0
HWI-EAS313:8:65:992:914#0	16	chr1	520	3	42M	*	TCTGGGCATCAGTAAAAAAATGGTGGTTCCAGTCATTACATC	A?@5;@=?@>@>ABABBBBBBA>A:>BA@?AABBBBBACC@;	NM:i:0
HWI-EAS313:8:10:620:686#0	0	chr1	559	3	42M	*	ATCAAGTCCACAGTTATTACTGAGAAAACCTGATCAGTTTAT	BB?BBAAB6A@BC;AABBB@B@BA@@@AB@AB>A>A@;AAAB	NM:i:0
HWI-EAS313:8:27:1782:1073#0	16	chr1	578	255	27M411N15M	CTGAGAAAACCTGATCAGTTTATGCAGAATGTTTTGTTTTTC	AA=@8@BB<A=A<?ABB@B@BBB=AA6BBB?BBBBBBBBBBB	NM:i:0	XS:A:+	NS:i:0
HWI-EAS313:8:43:783:738#0	0	chr1	1052	255	42M	*	AAAACAACAGGAAAAATTCTGTGTCGTTCGCCTGAAATATTT	:ABCBACBBCCBBBB?CB=A<>>ABB@B@@<BBB?BBBBBBB	NM:i:0
HWI-EAS313:8:78:753:514#0	0	chr1	1052	255	42M	*	AAAACAACAGGAAAAATTCTGTGTCGTTCGCCTGAAATATTT	AAA=CCAABACBC?BCA9<@@4B=<BB??C=?ABBBCBBB>7	NM:i:0
HWI-EAS313:8:17:949:19#0	0	chr1	1057	255	42M	*	AACAGGAAAAATTCTGTGTCGTTCGCCTGAAATATTTGCTTC	?CBCBCB=BBBCCBBB3?@@AB?AC8ABBBBBB@B@@@/B@9	NM:i:0

I get a BAM output and would like to know if this would be a true binary representative of the original SAM file?

any thoughts or suggestions?
Siva

[john_mu]

Yes, you can ignore that warning.

I'm not sure why the ID tag is missing... it should be

ID:Tophat

[Siva]

As a followup: I work with maize which has 10 chromosomes. So when it says "11 sequences loaded" does it refer to chromosomes 1 through 10 and chr Unknown? There is a chunk of sequences in the maize genome that has not been assigned to any chromosome.

Siva

[john_mu]

I am only familiar with the SAM format... not too familiar with the SAM to BAM conversion at this stage. So, I can't help you there, in case I give the wrong information.

But your reasoning sounds reasonable, since I believe the unmapped reads at denoted as a "*".

[Siva]

Yes, you can ignore that warning.

I'm not sure why the ID tag is missing... it should be

ID:Tophat

Thanks John, I too don't know why the tag ID is missing.

[jmarshall]

The missing ID: tag is a small bug in tophat. Mostly harmless, but it will prevent processing the SAM file with picard, even with VALIDATION_STRINGENCY=SILENT, or other tools that expect headers to be syntactically correct. It's trivial to fix in tophat.py (line 1006 in 1.0.13).

./../../samtools-0.1.7a/samtools import ../../../ZmB73_AGPv1_genome.fasta.fai accepted_hits.sam accepted_hits_complete.bam
The tag [ID] required for [PG] not present.
[sam_header_read2] 11 sequences loaded

The 11 here comes from the number of lines in your ZmB73_AGPv1_genome.fasta.fai file. This won't include the "*" denoting unmapped reads; rather, as you surmised, your reference presumably contains an extra "chromosome" labelled Unknown or so, containing those unassigned sequences.

-- John

[xue.vin]

Hi I am having a similar problem using picard to parse a BAM file from tophat. Is there anyway around it?

I get an error similar to this one when I try to validate the BAM file
Exception in thread "main" net.sf.samtools.SAMFormatException: Error parsing SAM header. Problem parsing @PG key:value pair. Line:
@PG TopHat VN:1.0.13

[mrawlins]

xue.vin,
The way to solve the problem with parsing the BAM in Picard would be to edit the SAM file, changing the line
@PG TopHat VN:1.0.13
to
@PG ID:TopHat VN:1.0.13

This can be done with a text editor, with sed/awk/perl, or by editing the TopHat source code to include the ID: on the @PG line. The line is at the top of the file. The best way to do this will depend on what system you're using, how big the SAM file is, and how comfortable you are with programming. If it's not a SAM, but a BAM, then your best bet is editing the TopHat source, since the BAM files use fairly complicated compression.

-Mitch

[xue.vin]

Thank You for the quick reply. Your suggestion was very helpful.

-Vincent

[shurjo]

The missing ID: tag is a small bug in tophat. Mostly harmless, but it will prevent processing the SAM file with picard, even with VALIDATION_STRINGENCY=SILENT, or other tools that expect headers to be syntactically correct. It's trivial to fix in tophat.py (line 1006 in 1.0.13).

./../../samtools-0.1.7a/samtools import ../../../ZmB73_AGPv1_genome.fasta.fai accepted_hits.sam accepted_hits_complete.bam
The tag [ID] required for [PG] not present.
[sam_header_read2] 11 sequences loaded

-- John

Does anyone know if this has been fixed in Tophat1.0.14?

[shurjo]

Just checked the tophat.py script, looks like it has. Maybe Picard will now work with the TopHat SAM files.....

[t.wieland]

does it also produce the @SQ entries now?
because some of the picard tools need them. I tried to add them, but it seems that I made a mistake somewhere.

[regyre]

No it does not.

Since I'm having it part of a workflow, and got bored to have to edit the header manually, I've created a patch file for tophat v1.0.14 (attached). It's probably not optimal, but it does add the SQ fields to the header.

To apply that patch to a fresh new installation:

1. unpack tophat-1.0.14.tar.gz
2. copy the attached patch to the tophat-1.0.14 directory
3. unzip the patch
4. verify that the patch works

   patch --dry-run -p1 -i tophat-1.0.14.py.patch

5. apply it

   patch -p1 -i tophat-1.0.14.py.patch

6. install tophat

   ./configure --prefix=`pwd`/../tophat
   make
   make install

To apply it to an existing installation, do the step 2 to 5 above and then do

make clean
make
make install

I hope this helps. It did for me :-)

I'll send the patch to Cole Trapnell (TopHat author) too, so that it does not get lost (and hopefully he will find it sensible).