Filter mpileup for high depth of coverage without bcftools/vcfutils

[Andrew Beckerman]

Hi

I am working with Illumina paired end read data, several populations. I am creating an mpileup from several .bam files, each from a different population. I would like to exclude candidate snp's where coverage in ANY one of the populations is ABOVE a read depth threshold.

To start, focusing on one scaffold, and just two of the populations, I would use

samtools mpileup -r scaffold_1 -D B1.20.bam D8.20.bam > s1B1D8.mpileup

Typically, I think one would now use the pipeline of mpileup and bcftools/vcfutils.pl setting -D, as in vcfutils.pl varFilter -D 2000.

However, I need to retain an mpileup file.

I can use the -D option in mpileup to retain a column with some measure of the depth. However, it is not clear to me what it is with several samples, as it does not match exactly the minimum or average for the bam files included, and seems to only return a single value even with more than one bam.

Any ideas, suggestions and insight would be most appreciated.
Andrew

[Andrew Beckerman]

Perhaps a solution myself

# generate mpilup, say with two bams

samtools mpileup -B a.bam b.bam > out.mpileup

# Then use awk to filter on the depth of coverage columns (i.e. $4 and $7 for two bams)
# i.e. with a depth of 100 max in any depth column

awk '$4 <= 100 && $7 <= 100' out.mpileup > sifted.mpileup