[Annovar] - "cannot write to outfile"

[Opio]

Hi,

I'm pretty new to bioinformatics.
I'm training on a set of 2 WES (whole exomes sequencing) to get introduced to all those steps.
I followed the GATK good practices, and I'm with a joint VCF file, and then I want to filter and annotate it with annovar.
Before working on annovar, I chose to convert vcf to avinput.

For that I tried the following :

convert2annovar.pl -format vcf4 -allsample input.vcf -outfile STDOUT

And i have the following error (ubuntu 14.04 or 16.04) :

NOTICE: output files will be written to STDOUT.<samplename>.avinput
cannot write to outfile

It's not about rights, my colleage bioinformatician looked it up with me, it's all about how convert2annovar deals with the argument -allsamples to get a outfile per sample.
Otherwise convert2annovar works fine to me.

The only workaround I found is to use the parameter "-withfreq" which makes it work like a charm..

I can't find anything on google, have you any ideas about where that error could come from?

Best,

[GenoMax]

STDOUT is a reserved output channel. Generally that refers to the screen you are using to enter commands/get output. Try using a different name in your command:

convert2annovar.pl -format vcf4 -allsample input.vcf -outfile MyFile

[Markiyan]

Do not use stdin or stdout words for regular filenames.

The stdout and stdin are reserved filenames in the unix/perl world...

So avoid using them in regular file names.

Better do:

convert2annovar.pl -format vcf4 -allsample input.vcf -outfile output

and see what it tells you.

PS: have a look at:

https://stackoverflow.com/questions/122400/what-are-reserved-filenames-for-various-platforms

and:

https://msdn.microsoft.com/en-us/library/aa365247.aspx

(if any part of your integrated systems runs on windows)

[Opio]

Thank for your answers !

Actually I put that name for the forum only, the one on my command was different

/genetics/ongoing_work/test_avinput$ /genetics/apps/annovar/convert2annovar.pl -format vcf4 -allsample 78_14_TR_p_BWA_mem_PIC_conv_ord_mdup_group_recal_rawvariants_Joint_INDEL_recalibrated_extracted_filter_appl_flqg.vcf -outfile 78_14_TR_p_BWA_mem_PIC_conv_ord_mdup_group_recal_rawvariants_Joint_INDEL_recalibrated_extracted_filter_appl_flqg_avinput

And the result :

NOTICE: output files will be written to 78_14_TR_p_BWA_mem_PIC_conv_ord_mdup_group_recal_rawvariants_Joint_INDEL_recalibrated_extracted_filter_appl_flqg_avinput.<samplename>.avinput
1 cannot write to outfile

And the same as well with myfile

By the way my OS is Linux Ubuntu14.04

if ($chr =~ m/^#CHR/i) {		#format specification line
			$found_headerline and next;	#sometimes people concatenate multiple VCF files together without removing header lines in the combined file; we need to address this
			$found_headerline++;
			if ($allsample and @sample and not $withfreq) {
				
				if (not @sample and $withfreq) {
					die "Error: -withfreq argument is set, but the VCF file does not contain any sample\n";
				}
				
				for my $i (0 .. @sample-1) {
					open ($fhout[$i], ">$outfile.$sample[$i].avinput") or die "1 cannot write to outfile\n";
					#open ($fhout[$i], ">$outfile.$sample[$i].avinput");
					my $fhout = $fhout[$i];		#must use a temp variable for this to work
					$comment and @comment and print $fhout join ("\n", @comment), "\n", join ("\t", $chr, $start, $ID, $ref, $alt, $quality_score, $filter, $info, $format, $sample[$i]), "\n";
				}
			} else {
				my $fhout;
				if ($outfile) {
					open ($fhout, ">$outfile") or die "Error: cannot write to output file $outfile: $!\n";
					$fhout[0] = $fhout;
				} else {
					$fhout[0] = *STDOUT;
					$fhout = $fhout[0];
				}
				$comment and @comment and print $fhout join ("\n", @comment), "\n", join ("\t", $chr, $start, $ID, $ref, $alt, $quality_score, $filter, $info, $format, $sample[0]), "\n";
				if (@sample > 1) {
					if (not $allsample or not $withfreq) {
						print STDERR "WARNING to old ANNOVAR users: this program no longer does line-to-line conversion for multi-sample VCF files. If you want to include all variants in output, use '-format vcf4old' or use '-format vcf4 -allsample -withfreq' instead.\n";
					}
				}
			}
			$countsample = scalar (@sample);
			next;

As a complement, I focused on the part of the code of convert2annovar.pl which raised the error. But as a not PERList, I don't exactly know what $fhout does

[GenoMax]

You may not have write permission to this directory (/genetics/ongoing_work/test_avinput) based on that error. What if you try to save the output files somewhere else?

$fhout is designating a file handle (name) to write the output to.

[Opio]

Apprently it seems that I have it !

drwxr-xr-x 2 username group 4096 Oct 25 15:24 test_avinput
I just remplaced username and group in that line, but that's the good one.

Weirdly enough, I have another machine with ubuntu which is administrated by my (I'm root on it) and I have the same issue.

I then think that it may not be the issue :/

Is my file can cause that? That would be weird because it goes well with -allsample -withfreq but not without the -withfreq

.. i'm lost !

[GenoMax]

Sounds like this is an issue of some combination of those command options for Annovar. You may want to search GATK forums to see if someone has seen this before.

[Opio]

I tried with no success so far, but I'll keep look at it.

But my question can be obvious though : how do you do from a joint vcf to put it in annovar ? You have to have that step in your pipeline I guess?

[Markiyan]

Include the file name in the error message in convert2annovar.pl

We need to improve our exception handling a bit.
We should always include the name on the file which our program tries to open for writing in the error message. Also differentiate between file open and file writing errors and include os error output ($! in perl) in the die message.

To do that replace in convert2annovar.pl :

for my $i (0 .. @sample-1) {
open ($fhout[$i], ">$outfile.$sample[$i].avinput") or die "1 cannot write to outfile\n";


with:

for my $i (0 .. @sample-1) {
open ($fhout[$i], ">$outfile.$sample[$i].avinput") or die "Cannot open output file for writing: $outfile.$sample[$i].avinput\n$!\n";

Than, please make sure that ALL sample names are defined for that vcf file(s), and that each sample name is unique, and has NO SPACES, SLASHES, and other file name unfriendly characters!

[Opio]

Hi,
thank you very much for helping me.
So I tried your part of code and I have with the command :

convert2annovar.pl -format vcf4 -allsample 78_14_TR_p_BWA_mem_PIC_conv_ord_mdup_group_recal_rawvariants_Joint_INDEL_recalibrated_extracted_filter_appl_flqg.vcf -outfile example

the following result :

NOTICE: output files will be written to example.<samplename>.avinput
Cannot open output file for writing: example./media/romain/MYBOOK/romain_work/78_81/78_14_TR_p_BWA_mem_PIC_conv_ord_mdup.bam.avinput
No such file or directory

which is indeed.. true. That files doesn't exist (even though /media/romain/MYBOOK/romain_work/78_81/78_14_TR_p_BWA_mem_PIC_conv_ord_mdup.bam exists)

So it's calling a file which is a compound of the output name and the source name?

By the way, side question : my vcf files calls for sample which are a path, even if the path is correct and the file exists, convert2annovar will work locally on the vcf file, not on the path of the sample right?


I guess the fact it has "/" makes it unhealthy for annovar

[Opio]

removed the bad "sample" name in my vcf file

... and it works fine now. Actually it was calling indeed a file made of a path, you pointed right.

Thank you so much.