bfast temporary file problem

[sammy07]

Hi, I have a weird problem with the bfast match step on Fedora 14. It looks something like this:

bfast match -f refGenome.fna -r /home/proj/data/reads/4.fastq

...
************************************************************
Searching for main indexes...
Found 1 index (1 file).
Not using secondary indexes.
************************************************************
Reading in reference genome from refGenome.fna.nt.brg.
In total read 10 contigs for a total of 1000000000 bases
************************************************************
Reading /home/proj/data/reads/4.fastq into a temp file.
************************************************************
In function "OpenTmpGZFile": Fatal Error[OpenFileError]. Variable/Value: ./.bfast.tmp.2QJsyX.
Message: Could not open temporary file.
The file stream error was:: No such file or directory
***** Exiting due to errors *****

Its not about the writing rights of the tmpDir, as full writing rights don't seem to help. Do I miss something obvious..?
Thanks in advance.

[drio]

Do you have enough space in the filesystem where your current dir is mounted?

[sammy07]

Yes, definitely...

[drio]

Are you running this interactively or you send the job to a cluster node?
The reason why I am asking is because when the later, sometimes the filesystems
are not available or the working directory changes.

The error is coming from BLib.c (line 549). Capture the error code from
zlib's gzdopen and print it. Then check with the list of error codes from zlib to see if
you get an explanation of what is happening.

[sammy07]

Hi drio, thanks for your help! I am testing the single commands interactively. The filesystem is mounted via nfs; could this be a problem? However, it seemed to work before; no idea what has changed.

Thanks for the hint about BLib.c (question: how do you find out where the error is? via something like "for file in ls: echo filename; cat filename | grep errormessage"?). It seems that the temporary file is created but cannot be handled by gzdopen and the file pointer fp is NULL?

I have no experience with capturing error code, but doesnt the message "The file stream error was:: No such file or directory" contain the error description which belongs to the error code contained in the global "errno"?

[nilshomer]

For some reason, you are not allowed to open a new file in that directory. That could be for a number of reasons, including space, permissions, # of file handles open at one time, etc. I would instead use the "-T" option to specify the temporary directory to a place with a lot space and where you have permission to create new files.

[sammy07]

Hi Nils, thank you. I already tried to use -T to specify a Temp directory in my home folder with full writing permissions (777), it didn't help. At least now I know what it is all about; if I find out something, I will let you know.

[drio]

I suggest you ask your friendly sysadmin. I am pretty sure it has to with your environment.

[athomma]

I have the same problem with bfast match and Fedora 14. With Fedora 13 the bfast steps performed well.

This is my error message.

************************************************************
Searching for main indexes...
Found 10 index (10 total files).
Not using secondary indexes.
************************************************************
Reading in reference genome from /tmp/ref_Genome.fna
In total read 1 contigs for a total of 3218031 bases
************************************************************
Reading /tmp/reads.fastq into a temp file.
************************************************************
In function "OpenTmpGZFile": Fatal Error[OpenFileError]. Variable/Value: /tmp/.bfast.tmp.3fC7L9.
Message: Could not open temporary file.
The file stream error was:: No such file or directory
 ***** Exiting due to errors *****
************************************************************
Done

Some additional ideas and search for the problem hints that the filename might be wrong:

# ls -l /tmp/
-rw-------   1          0 Jan  3 15:41 .bfast.tmp.g0h0r7

This file is created before the error.

I assume somehow the file created has a different name than the stream which was read.

[drio]

Thanks for bringing this up (on this one thread). Let me see if I can install a VM with fedora14 and reproduce the problem.

[drio]

sammy07, athomma,

Could you please try the git version? That should fix the issue.

$ git clone git://bfast.git.sourceforge.net/gitroot/bfast/bfast
$ sh ./autogen.sh && ./configure && make

Thanks.

[athomma]

Thanks

Thank you for the solution.

The git version works for me.

Bye

[pdblood]

Hi,

We are experiencing the same problem described in this post:

Reading /bessemer/fatakias/p2_reads.708.fastq into a temp file.
************************************************************
^MIn function "OpenTmpGZFile": Fatal Error[OpenFileError]. Variable/Value: ./.bfast.tmp.mXabZb.
Message: Could not open temporary file.
The file stream error was:: No such file or directory
***** Exiting due to errors *****
************************************************************

The same thing happens no matter what filesystem we run from (i.e. permissions and space are not the problem). We have used the latest git version, but this does not solve the problem. We are running on an SGI Altix UV system running SLES 11 with an SGI-modified linux kernel. More info about the machine can be found here: blacklight.psc.edu

[cheney]

I found the same error info in a CentOS 5.5 server(4-way,xeon, 64G ram), with galaxy integrated, when make check. I had all the related packages rebuild from src files.
I found that 'bfast temporary OpenFileError OpenTmpGZFile openfile error' only accept bfast index -depth=0. and this error was due to the index step. it can also make the match work with the index files built in another machine .
Hope this can be helpful.
Index code

 btestindexes -A 1 -r 50 -s 20 -S 10000  -l 24 -a 0 -w 30 -n 20 -E 2  -f hg19.fa
 bfast index -A 1 -d 0 -w 15 -n 1  -m 11111111011110111111111111 -t 1 -f hg19.fa -T /media/_people2/tmp/

http://hi.baidu.com/sichaocourse/blog/item/fec244394ffe29ef3b87ce49.html

[houliping]

I have the same problem with the newest version of bfast (0.7.0b).

The problem was solved by following the instruction I found in this link.

http://sourceforge.net/mailarchive/forum.php?thread_name=CAEFx-FywSU1GavaVXH9YYp1gT0L91bGhWEjEDnrXYjzfNC60mA%40mail.gmail.com&forum_name=bfast-help

What I did is to change the line 553 in BLib.c file from:

NULL == (fp = gzdopen(fd, ZLIB_VERNUM >= 0x1250 ? "ab" : "wb+"))) {

to:

NULL == (fp = gzdopen(fd, "ab"))) {

By the way, my zlib version is 1.26.