Issues with Fst from vcftools

[deancastillo]

I am having issues getting Fst values from vcftools that make sense (in range from 0 to 1). I think there is an issue with how alleles are being called (maybe issue with the vcf file?).
I have 2 populations with 4 individuals from each population (8 lines total). I have vcf files of each population separately (4 individuals in each of 2 files) and then a vcf file with all 8 individuals. When I look at allele counts for all 8 individuals it is incorrect (not the sum of allele counts if I add counts from the 2 files that contain only 4 individuals)
Below are allele counts for a single gene. Allele counts for pos 702 look correct to me and so do counts for pos 51 (assuming pop2 is all A), but the other two sites are not correct even though site 12 has variants in both populations.
Does someone have an idea of what may be going wrong?
Each vcf file was generated from mpileup with the exact same command just differing the number and identity of the sorted .bam files

Pop1
CHROM POS N_ALLELES N_CHR {ALLELE:COUNT}
Dpse-Amy3-PA 12 2 8 T:3 G:5
Dpse-Amy3-PA 51 2 8 A:0 G:8
Dpse-Amy3-PA 702 2 8 T:4 C:4

Pop2
CHROM POS N_ALLELES N_CHR {ALLELE:COUNT}
Dpse-Amy3-PA 10 2 8 A:3 C:5
Dpse-Amy3-PA 12 2 8 T:3 G:5
Dpse-Amy3-PA 702 2 8 T:2 C:6

Both Pops
CHROM POS N_ALLELES N_CHR {ALLELE:COUNT}
Dpse-Amy3-PA 10 2 16 A:0 C:16
Dpse-Amy3-PA 12 2 16 T:0 G:16
Dpse-Amy3-PA 51 2 16 A:8 G:8
Dpse-Amy3-PA 702 2 16 T:6 C:10

[bio_informatics]

If you do not get a reply here, you may use their [VCFtools'] mailing list for help. They will reply.

[dschika]

I also think [VCFtools] mailing list is best for questions about VCFtools.

Nevertheless, how were the vcf files generated? Did you merge your single individual vcf files into one containing all 8 individuals?
Have you had a look at the vcf files to see what is reported there e.g. at position 12?

Btw: Fst values calculated with VCFtools can be slightly negative.

[deancastillo]

The bcf files were generated using samtools mpileup. In the mpileup command I listed all 8 bam files, one bam file per sample (or 4 for the two smaller vcf files). I then converted to vcf with bcftols -view.

In the vcf files the allele counts are the same as the output from vcftools.
At position 12 for the whole sample all individuals are 1/1 meaning they are homozygous for the "G" but this is not true according to the smaller vcf files

I will also try the vcf mailing list, thank you both for this suggestion

[dschika]

In the vcf files the allele counts are the same as the output from vcftools.

If the numbers in the vcf file are already different, you do not have issues with VCFtools.

Do you apply any filters during vcf file generation, e.g. minimum coverage per individual?

[clarissaboschi]

no Fst result

I am also having problems to analyze my vcf file with VCFtools (to get Fst statistics).

I have one vcf file from samtools of multiple samples (2 different popultions, 28 samples)
I used the command line:

vcftools --vcf  inputfile.vcf --weir-fst-pop pop1_members.pop --weir-fst-pop pop2_members.pop

But in my results there is no value for Fst, only "-nan"

cat out.weir.fst | head
CHROM POS WEIR_AND_COCKERHAM_FST
Chr10 19 -nan
Chr10 31 -nan
Chr10 133 -nan
Chr10 247 -nan
....

What could be the issue? My vcf file seems ok.

[dschika]

Are the names of the samples in the vcf and the population files exactly the same?

[clarissaboschi]

Dear dschika,
This was the issue!! Thanks very much for your help

Clarissa