samtools bcf to vcf

[Anjali]

Hi
I have a bcf file generated using bcftools(samtools), which I further wanted to convert to vcf format. I used the following command:

bcftools view ../bcf/test.bcf | vcfutils.pl varFilter -D 8000 > test.vcf

The corresponding vcf file only generated the header and not the body.

I have used this command 2 months before on a different dataset and it worked well. I have the same Human fasta index file as a refernce. I can not figure out what is the problem now.

Can anyone soon help me??

[Anjali]

Is there no one who can give an idea over this concern ??

[xied75]

What do you get if you only:

bcftools view ../bcf/test.bcf > test.vcf

[swbarnes2]

Well, what happens when you don't apply the -D8000 filters to vcfutils? What happens when you don't pipe through vcfutils at all?

[Anjali]

when I don't use vcfutils at all and use only the bcftools view command I still get a large vcf file which is not readable
I am sure it is doing the allignment but I still do not get the variants.

Can you please explain me the reason of this problem?

[xied75]

Maybe you can run:
hexdump -C yourbcf.bcf | more
and
hexdump -C your-not-readable-vcf.vcf | more
Paste here so we can have a look, one screen is enough.

[Anjali]

after putting this hexdump command to the bcf file, it was the following image :

00000000 1f 8b 08 04 00 00 00 00 00 ff 06 00 42 43 02 00 |............BC..|
00000010 a3 03 b5 56 5d 6f da 30 14 e5 61 2f db af b0 c6 |...V]o.0..a/....|
00000020 0b 95 02 25 81 ae ed 34 26 51 28 0c 89 76 94 b2 |...%...4&Q(..v..|
00000030 6e 7b 59 65 92 0b b1 96 d8 a9 ed 94 f5 2f ec 7d |n{Ye........./.}|
00000040 ff 77 d7 0e a1 4d ca 5a b4 69 2f 97 60 72 cf 39 |.w...M.Z.i/.`r.9|
00000050 f7 d3 9c f4 06 2f 7e 56 2a 15 3f 94 ae 31 9e 31 |...../~V*.?..1.1|

[Anjali]

for the vcf file it is:


00000000 23 23 66 69 6c 65 66 6f 72 6d 61 74 3d 56 43 46 |##fileformat=VCF|
00000010 76 34 2e 31 0a 23 23 73 61 6d 74 6f 6f 6c 73 56 |v4.1.##samtoolsV|
00000020 65 72 73 69 6f 6e 3d 30 2e 31 2e 31 38 20 28 72 |ersion=0.1.18 (r|
00000030 39 38 32 3a 32 39 35 29 0a 23 23 49 4e 46 4f 3d |982:295).##INFO=|
00000040 3c 49 44 3d 44 50 2c 4e 75 6d 62 65 72 3d 31 2c |<ID=DP,Number=1,|
00000050 54 79 70 65 3d 49 6e 74 65 67 65 72 2c 44 65 73 |Type=Integer,Des|
00000060 63 72 69 70 74 69 6f 6e 3d 22 52 61 77 20 72 65 |cription="Raw re|
00000070 61 64 20 64 65 70 74 68 22 3e 0a 23 23 49 4e 46 |ad depth">.##INF|
00000080 4f 3d 3c 49 44 3d 44 50 34 2c 4e 75 6d 62 65 72 |O=<ID=DP4,Number|
00000090 3d 34 2c 54 79 70 65 3d 49 6e 74 65 67 65 72 2c |=4,Type=Integer,|
000000a0 44 65 73 63 72 69 70 74 69 6f 6e 3d 22 23 20 68 |Description="# h|
000000b0 69 67 68 2d 71 75 61 6c 69 74 79 20 72 65 66 2d |igh-quality ref-|
000000c0 66 6f 72 77 61 72 64 20 62 61 73 65 73 2c 20 72 |forward bases, r|
000000d0 65 66 2d 72 65 76 65 72 73 65 2c 20 61 6c 74 2d |ef-reverse, alt-|
000000e0 66 6f 72 77 61 72 64 20 61 6e 64 20 61 6c 74 2d |forward and alt-|
000000f0 72 65 76 65 72 73 65 20 62 61 73 65 73 22 3e 0a |reverse bases">.|
00000100 23 23 49 4e 46 4f 3d 3c 49 44 3d 4d 51 2c 4e 75 |##INFO=<ID=MQ,Nu|
00000110 6d 62 65 72 3d 31 2c 54 79 70 65 3d 49 6e 74 65 |mber=1,Type=Inte|
00000120 67 65 72 2c 44 65 73 63 72 69 70 74 69 6f 6e 3d |ger,Description=|
00000130 22 52 6f 6f 74 2d 6d 65 61 6e 2d 73 71 75 61 72 |"Root-mean-squar|
00000140 65 20 6d 61 70 70 69 6e 67 20 71 75 61 6c 69 74 |e mapping qualit|
00000150 79 20 6f 66 20 63 6f 76 65 72 69 6e 67 20 72 65 |y of covering re|
00000160 61 64 73 22 3e 0a 23 23 49 4e 46 4f 3d 3c 49 44 |ads">.##INFO=<ID|

[xied75]

So your bcf is binary and the header looks ok. Your vcf is text why you say it's un-readable? Are there lines after the vcf header?

[Anjali]

The vcf file without the use of varfilter is a 20GB file. I can not open it. I can see the header as well as the lower part when I scroll down the linux window, after header it looks like this :

00000b40 2c 31 2c 30 3b 4d 51 3d 36 30 3b 46 51 3d 2d 33 |,1,0;MQ=60;FQ=-3|
00000b50 30 09 47 54 3a 50 4c 3a 47 51 09 31 2f 31 3a 33 |0.GT:PL:GQ.1/1:3|
00000b60 37 2c 33 2c 30 3a 34 0a 63 68 72 31 09 31 30 30 |7,3,0:4.chr1.100|
00000b70 30 36 09 2e 09 4e 09 43 09 37 2e 38 09 2e 09 44 |06...N.C.7.8...D|
00000b80 50 3d 31 3b 41 46 31 3d 31 3b 41 43 31 3d 32 3b |P=1;AF1=1;AC1=2;|
00000b90 44 50 34 3d 30 2c 30 2c 31 2c 30 3b 4d 51 3d 36 |DP4=0,0,1,0;MQ=6|
00000ba0 30 3b 46 51 3d 2d 33 30 09 47 54 3a 50 4c 3a 47 |0;FQ=-30.GT:PL:G|
00000bb0 51 09 31 2f 31 3a 33 37 2c 33 2c 30 3a 34 0a 63 |Q.1/1:37,3,0:4.c|

it looks fine,, but what is the problem, when i use the vcfutils and a varfilter,, bcz i want only certain SNPS

[xied75]

Then I guess your -D 8000 filtered everything out. Try reduce that.

[Anjali]

I did this in samtools, where when during alignment I give the mpileup command it says to set the read depth to 8000.

Before when the varFilter was set to 8000 it was only a header in vcf file and no SNPs, but now when I tried with -D 100 , the file did not run.
The vcf file is 0 in size

[xied75]

It seems you have N in the vcf, read this thread:

http://seqanswers.com/forums/showthread.php?t=17486&highlight=vcfutils+varfilter

[Anjali]

Yes, you are right. I already saw this conversation before, but I think there was a problem with the version of samtools they were using,

I have the samtools version 1.1.18 and it worked well before but now it is having problem with the latest dataset.

may be you can suggest me another reference file to try aligning again