Problems installing bcftools and fixref plugin for using Sanger Imputation Service

[LeonDK]

Dear all,

I am trying to use the Sanger imputation service (https://imputation.sanger.ac.uk/), but I get the following error:
"Reference allele mismatch at 1:752566 .. REF_SEQ:'G' vs VCF:'A'"

According to this link https://imputation.sanger.ac.uk/?resources=1, this can be fixed by installing bcftools and using the fixref plugin available here https://github.com/pd3/bcftools/blob...ugins/fixref.c

However, after installing bcftools, there are no plugins:

bcftools plugin -vl
plugin directory /usr/local/libexec/bcftools .. ok

No functional bcftools plugins were found. The environment variable BCFTOOLS_PLUGINS is not set.

I have downloaded fixref.c and placed it in /usr/local/libexec/bcftools but still nok luck. If I try to compile fixref.c I get:

gcc fixref.c 
fixref.c:84:22: fatal error: bcftools.h: No such file or directory
 #include "bcftools.h"
                      ^
compilation terminated.

Despite bcftools being installed:

which bcftools 
/usr/local/bin/bcftools

Not really sure what to do here?

[GenoMax]

Did you compile bcftools yourself? If so you should have the bcftools.h header file available in the source code that you downloaded. You can edit the fixref.c file to include full path to the bcftools.h file and see if you can get the plug-in to compile.

[LeonDK]

Okay, so I got the plugins to install, by going to https://samtools.github.io/bcftools/ and then issuing the following commands:

 git clone --branch=develop git://github.com/samtools/htslib.git
git clone --branch=develop git://github.com/samtools/bcftools.git
git clone --branch=develop git://github.com/samtools/samtools.git
cd bcftools; make
cd ../samtools; make

Followed by:

cd ../bcftools; sudo make install
cd ../samtools; sudo make install

Then I get the plugins:

bcftools plugin -lv | head

-- GTisec --
Count genotype intersections across all possible sample subsets in a vcf file.

-- color-chrs --
Color shared chromosomal segments, requires phased GTs.

-- counts --
A minimal plugin which counts number of samples, SNPs,
INDELs, MNPs and total number of sites.

But I still cannot compile the fixref.c plugin:

cd /usr/local/libexec/bcftools/ ; sudo gcc -shared -o fixref.so -fPIC fixref.c
fixref.c:84:22: fatal error: bcftools.h: No such file or directory
 #include "bcftools.h"
                      ^
compilation terminated.

[GenoMax]

See where the bcftools source was put locally. bcftools.h should be in the top level directory. Edit the fixref.x source to include path to that file.

#include "/path_to/bcftools.h"

[LeonDK]

See where the bcftools source was put locally. bcftools.h should be in the top level directory. Edit the fixref.x source to include path to that file.

#include "/path_to/bcftools.h"

Thank you, that worked with respect to getting fixref.c compiled into fixref.so, like so:

sudo gcc -shared -o fixref.so -fPIC fixref.c

however, unfortunately I am not quite there yet:

$ bcftools plugin fixref
Could not initialize fixref, version string not found

[GenoMax]

Check the syntax for using the plugin at the link you included above. It appears to go something like this.

bcftools +fixref file.bcf -- -f ref.fa

[LeonDK]

Check the syntax for using the plugin at the link you included above. It appears to go something like this.

bcftools +fixref file.bcf -- -f ref.fa

Thanks a lot for input GenoMax! It's getting late in my part of the world, i will continue working on the issue tomorrow...

[LeonDK]

Check the syntax for using the plugin at the link you included above. It appears to go something like this.

bcftools +fixref file.bcf -- -f ref.fa

It's not flying, despite using the stated syntax, I still get

Could not initialize fixref, version string not found

I cannot find any elaboration on the fixref plugin on Sanger. I have tried to look in the source code, but I cannot seem to identify the error.

Thanks for input, I will try to mail the source code author and ask.

[LeonDK]

I have checked out the "Plugins troubleshooting:" at https://samtools.github.io/bcftools/bcftools.html with no luck.

So far unfortunately, I am stuck. I have written Sanger and the author and I am currently awaiting an answer. Should I get one, I will return to this thread with it for any future reference...

[jmarshall]

Thank you, that worked with respect to getting fixref.c compiled into fixref.so, like so:

sudo gcc -shared -o fixref.so -fPIC fixref.c

That only partially compiled it (to be precise, it did not link it).

To compile this plugin properly, you need to reproduce the commands that are used to compile the other plugins. So you should drop fixref.c into your bcftools-x.y.z/plugins source directory, and use a command like:

gcc -fPIC -shared -O2 -I. -I../htslib -o plugins/fixref.so version.c plugins/fixref.c

to compile it (which you run from the parent bcftools-x.y.z source code directory).

[LeonDK]

That only partially compiled it (to be precise, it did not link it).

To compile this plugin properly, you need to reproduce the commands that are used to compile the other plugins. So you should drop fixref.c into your bcftools-x.y.z/plugins source directory, and use a command like:

gcc -fPIC -shared -O2 -I. -I../htslib -o plugins/fixref.so version.c plugins/fixref.c

to compile it (which you run from the parent bcftools-x.y.z source code directory).

You sir, are my new hero!

$ cd ~/tmp/bcftools/
$ wget https://raw.githubusercontent.com/pd3/bcftools/develop/plugins/fixref.c
$ gcc -fPIC -shared -O2 -I . -I ../htslib/ -o fixref.so version.c fixref.c
$ mv fixref.so /usr/local/libexec/bcftools/
$ bcftools plugin fixref -h

About: This tool helps to determine and fix strand orientation.
       Currently the following modes are recognised:
           flip  .. flips non-ambiguous SNPs and ignores the rest
           stats .. converts from Illumina TOP strand to fwd
           top   .. converts from Illumina TOP strand to fwd

Might I suggest that a little more information on how to properly install the fixref-plugin is added? Alternatively it could come bundled with bcftools, now that it is explicitly suggested as part of the Sanger Imputation Service?

Thanks again!

[jmarshall]

I expect that it will indeed be bundled within the next bcftools release.

[Petr]

Glad that it worked for you in the end.

Just a note, you don't need to invoke gcc directly, here is a slightly simpler version:

$ cd ~/tmp/bcftools
$ wget -O plugins/fixref.c https://raw.githubusercontent.com/pd3/bcftools/develop/plugins/fixref.c
$ make
$ mv plugins/fixref.so /usr/local/libexec/bcftools/
$ bcftools +fixref -h

[Petr]

I see that people are repeatedly struggling with the installation. Because of the rapid development of bcftools and htslib, best is to get the latest development version of both using the instructions described at this page http://samtools.github.io/bcftools.

In short:

    git clone git://github.com/samtools/htslib.git
    git clone git://github.com/samtools/bcftools.git
    cd bcftools; make

The clone command above is used to create a local copy of the remote repository and needs to be run only once. Whenever the latest snapshot from github is needed, use instead the pull command:

    cd htslib; git pull
    cd ../bcftools; git pull
    make clean
    make

IMPORTANT: In order to use the BCFtools plugins, this environment variable must be set to point to the correct location:

    export BCFTOOLS_PLUGINS=/path/to/bcftools/plugins

Verify by running

    /path/to/bcftools/bcftools plugin -l
    /path/to/bcftools/bcftools +fixref -h

The first command should print the list of available plugins (all *.so files in the plugins directory) and the second should print the usage help text for fixref.