If one read of a pair is unmapped, is the pair improper?

[Matt.Ralston]

Hello everyone, I have a simple question about generic paired-end Illumina data. If one read of a pair is unmapped, does this automatically mean that the pair is improper and is (un)flagged in a SAM file?

Thanks!

[Brian Bushnell]

No. An improper pair has both reads mapped, but not in the correct position/orientation for pairing.

But you may be asking about the 0x2 bit, which is poorly defined. According to the spec:
"0x2 each segment properly aligned according to the aligner"

In practice, paired reads where both mapped in the correct orientation and distance get the 0x2 bit set, and otherwise it is not set. By definition, it should be set for single-ended mapped reads, but that is not usually done in practice.

[westerman]

Brian: Which samtools spec are you referring to. When I read:

http://samtools.sourceforge.net/samtools.shtml

Then I get:

0x0002	P	the read is mapped in a proper pair

Which seems clear to me.

[Brian Bushnell]

http://samtools.github.io/hts-specs/SAMv1.pdf

2. FLAG: bitwise FLAG. Each bit is explained in the following table:
Bit Description
0x1 template having multiple segments in sequencing
0x2 each segment properly aligned according to the aligner
0x4 segment unmapped
0x8 next segment in the template unmapped
0x10 SEQ being reverse complemented
0x20 SEQ of the next segment in the template being reversed
0x40 the rst segment in the template
0x80 the last segment in the template
0x100 secondary alignment
0x200 not passing quality controls
0x400 PCR or optical duplicate
0x800 supplementary alignment

I believe that is the official specification, but it's ambiguous about many things.

[westerman]

Brian: Thanks. I have glanced at your referenced page in the past but did not notice the discrepancy between its definitions of the flag bit and the ones in the samtools manual page (which, IMHO, are more clear.) I almost always use the samtools program to do my work so that is what I am familiar with. Since you have written at least one program -- the very nice BBMap -- then you are, rightfully, following the official specs.

From the samtools man page:

Flag	Chr	Description
0x0001	p	the read is paired in sequencing
0x0002	P	the read is mapped in a proper pair
0x0004	u	the query sequence itself is unmapped
0x0008	U	the mate is unmapped
0x0010	r	strand of the query (1 for reverse)
0x0020	R	strand of the mate
0x0040	1	the read is the first read in a pair
0x0080	2	the read is the second read in a pair
0x0100	s	the alignment is not primary
0x0200	f	the read fails platform/vendor quality checks
0x0400	d	the read is either a PCR or an optical duplicate