VarScan mpileup2snp --p-value affects genotype call

[Graham Etherington]

Hi,
I'm using VarScan.v2.3.2 to analyse genotypes of susceptible and resistant plants. Sample1 is susceptible and Sample2 is resistant to a given pathogen.
I've noticed that when using mpileup2snp, changing the --p-value can change both the genotype call and the genotype quality.
I've included the command line and output of one of many positions where this happens VCF output followed by non-VCF output.
Command line 1 using --p-value 0.01:

java -jar VarScan.v2.3.2.jar mpileup2snp in.pileup --min-coverage 8 --min-reads2 2 --min-var-freq 0.01 --min-avg-qual 15 [B]--p-value 0.01[/B] --strand-filter 0 --output-vcf --variants 0 > out_p0.01.vcf

VCF Output:
chr01 2827 . A G . PASS ADP=57;WT=1;HET=1;HOM=0;NC=0 GT:GQ:SDPP:RD:AD:FREQ:PVAL:RBQ:ABQ:RDF:RDR:ADF:ADR 0/0:4:57:48:43:5:10.42%:2.8014E-2:37:24:20:23:5:0 0/1:4:85:67:53:14:20.9%:2.8637E-5:37:19:29:24:12:2

Tab-delim output
chr01 2827 A G R:115:96:19:16.52%:8.4889E-7 Pass:29:24:12:2:1E0 1 1 0 0 A:48:43:5:10.42%:2.8014E-2 R:67:53:14:20.9%:2.8637E-5

Command line 2 using --p-value 0.05:

java -jar VarScan.v2.3.2.jar mpileup2snp in.pileup --min-coverage 8 --min-reads2 2 --min-var-freq 0.01 --min-avg-qual 15 [B]--p-value 0.05[/B] --strand-filter 0 --output-vcf --variants 0 > out_p0.05.vcf

VCF Output:
chr01 2827 . A G . PASS ADP=57;WT=0;HET=2;HOM=0;NC=0 GT:GQ:SDPP:RD:AD:FREQ:PVAL:RBQ:ABQ:RDF:RDR:ADF:ADR 0/1:1:57:48:43:5:10.42%:2.8014E-2:37:24:20:23:5:0 0/1:4:85:67:53:14:20.9%:2.8637E-5:37:19:29:24:12:2

Tab-delim output
chr01 2827 A G R:115:96:19:16.52%:8.4889E-7 Pass:49:47:17:2:1E0 0 2 0 0 R:48:43:5:10.42%:2.8014E-2 R:67:53:14:20.9%:2.8637E-5

Here is the MPileup for that position:
chr01 2827 a 57 ,.,g,.,,,,.,,,,,......,,,,,,,,.GGGGGGGGGGGGG,,,.......... BD<!IIGHIIIIIIIAIGHHBFEHIIBCHHI9+&&*>-&9'<2-G=IHF8IIBI@IH 85 ,,GG,,,,,,,,,g,.,,.,,g,,,,,.,.........gGGGGGGGGGGGGG..........GGGGGGG......ggggg.,,,^2, IH9:IIGHIIG=I!IIDIGGG7IGF;II>GIIHBBGHH0#13427+/&(&41HIIEEDIIIH'*4*<*3HHGID@/'%%&DF#G#


So, the key differences in the VCF outputs are that the genotype and genotype quality differ with different p-values.
In the native VarScan tab-delimited output the differences are that StrandFilter, samplesHet/samplesRef, and the consensus genotypes are different.

So, I guess my question is this - I presumed that the --p-value option was just a filter, filtering out sites that have a FET lower than that provided.
It appears that the p-value does a lot more than that and influences which bases are included in the genotype calling.
Could anyone explain what's happening here? Also, how is Genotype Quality (FORMAT=<ID=GQ>) calculated in the VCF file?

Many thanks,
Graham

[Graham Etherington]

Dan Koboldt has kindly given a full explanation of how the supplied p-value can affect genotype calls and genotype quality calculation at the VarScan Help forum at:

http://sourceforge.net/projects/varscan/forums/forum/1073559/topic/6024166

[dkoboldt]

Graham, thanks for the link!