BFAST match problem

[blu78]

I have tried

bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -A 1 -n 8 > matchF3_10Mb 2> err_10Mb

but I still get the segmentation fault.

Indexes are created with the following commands:

bfast index -f contigs.fasta -A 1 -m 1111111111111111111111 -i 1 -w 12 -n 8
bfast index -f contigs.fasta -A 1 -m 111110100111110011111111111 -i 2 -w 12 -n 8
bfast index -f contigs.fasta -A 1 -m 10111111011001100011111000111111 -i 3 -w 12 -n 8
bfast index -f contigs.fasta -A 1 -m 1111111100101111000001100011111011 -i 4 -w 12 -n 8

...

bfast index -f contigs.fasta -A 1 -m 1110110001011010011100101111101111 -i 10 -w 12 -n 8

Thanks

[nilshomer]

Hi Nils,

Thanks for your reply.
Here is the command I use (on a much smaller set of reads now -- 1 000 000):

bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -i 1 -I 2,3 -A 1 -K 8 -M 512 -w 0 -n 8 > matchF3_10Mb 2> err_10Mb

I get the same results even if I skip secondary indexes and, as you suggest, I use the following command:

bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -i 1 1,2,3 -A 1 -K 8 -M 512 -w 0 -n 8 > matchF3_10Mb 2> err_10Mb

and even if I do not specify the primary indexes with -i (in that case I would use all my 10 indexes instead of just three).

What is kind of wierd is that in this latter case the segmentation fault occurs after searching index 4, while in the first and second case it occurs after searching index 1.

Any clue on what is going on?

Thanks again for your help...
Cheers

Try this:

bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -A 1 -n 8 > matchF3_10Mb 2> err_10Mb

Make sure that you have built all 10 indexes as recommended in the manual and use them as all as primary indexes. I also removed some options above as I would still recommend using the default parameters

It also looks like your stderr and stdout show that the process is not complete (meaning they don't show crashing on the "copying for next index search").

[blu78]

For completeness, here is the content of the stderr file obtained by $ cat err_10Mb

First two cases:

************************************************************
Checking input parameters supplied by the user ...
Validating fastaFileName contigs.fasta.
Validating readsFileName F3_10Mchunk.qfasta.
Validating tmpDir path ./.
**** Input arguments look good!
************************************************************
************************************************************
Printing Program Parameters:
programMode:                            [ExecuteProgram]
fastaFileName:                          contigs.fasta
mainIndexes                             1,2,3
secondaryIndexes                        [Not Using]
readsFileName:                          F3_10Mchunk.qfasta
offsets:                                [Using All]
loadAllIndexes:                         [Not Using]
compression:                            [Not Using]
space:                                  [Color Space]
startReadNum:                           1
endReadNum:                             2147483647
keySize:                                [Not Using]
maxKeyMatches:                          8
maxNumMatches:                          512
whichStrand:                            [Both Strands]
numThreads:                             8
queueLength:                            250000
tmpDir:                                 ./
timing:                                 [Not Using]
************************************************************
Searching for main indexes...
Found 3 index (3 total files).
Not using secondary indexes.
************************************************************
Reading in reference genome from contigs.fasta.cs.brg.
In total read 11 contigs for a total of 776592 bases
************************************************************
Reading F3_10Mchunk.qfasta into a temp file.
Will process 1000000 reads.
************************************************************
Searching index file 1/3 (index #1, bin #1)...
Reading index from contigs.fasta.cs.1.1.bif.
Read index from contigs.fasta.cs.1.1.bif.
Reads processed: 1000000
Cleaning up index.
Searching index file 1/3 (index #1, bin #1) complete...
Found 19503 matches.
************************************************************
Searching index file 2/3 (index #2, bin #1)...
Reading index from contigs.fasta.cs.2.1.bif.

Last case (with 10 indexes):

************************************************************
Checking input parameters supplied by the user ...
Validating fastaFileName contigs.fasta.
Validating readsFileName F3_10Mchunk.qfasta.
Validating tmpDir path ./.
**** Input arguments look good!
************************************************************
************************************************************
Printing Program Parameters:
programMode:                            [ExecuteProgram]
fastaFileName:                          contigs.fasta
mainIndexes                             [Auto-recognizing]
secondaryIndexes                        [Not Using]
readsFileName:                          F3_10Mchunk.qfasta
offsets:                                [Using All]
loadAllIndexes:                         [Not Using]
compression:                            [Not Using]
space:                                  [Color Space]
startReadNum:                           1
endReadNum:                             2147483647
keySize:                                [Not Using]
maxKeyMatches:                          8
maxNumMatches:                          512
whichStrand:                            [Both Strands]
numThreads:                             8
queueLength:                            250000
tmpDir:                                 ./
timing:                                 [Not Using]
************************************************************
Searching for main indexes...
Found 10 index (10 total files).
Not using secondary indexes.
************************************************************
Reading in reference genome from contigs.fasta.cs.brg.
In total read 11 contigs for a total of 776592 bases
************************************************************
Reading F3_10Mchunk.qfasta into a temp file.
Will process 1000000 reads.
************************************************************
Searching index file 1/10 (index #1, bin #1)...
Reading index from contigs.fasta.cs.1.1.bif.
Read index from contigs.fasta.cs.1.1.bif.
Reads processed: 1000000
Cleaning up index.
Searching index file 1/10 (index #1, bin #1) complete...
Found 19503 matches.
************************************************************
Searching index file 2/10 (index #2, bin #1)...
Reading index from contigs.fasta.cs.2.1.bif.
Read index from contigs.fasta.cs.2.1.bif.
Reads processed: 1000000
Cleaning up index.
Searching index file 2/10 (index #2, bin #1) complete...
Found 18896 matches.
************************************************************
Searching index file 3/10 (index #3, bin #1)...
Reading index from contigs.fasta.cs.3.1.bif.
Read index from contigs.fasta.cs.3.1.bif.
Reads processed: 1000000
Cleaning up index.
Searching index file 3/10 (index #3, bin #1) complete...
Found 14092 matches.
************************************************************
Searching index file 4/10 (index #4, bin #1)...
Reading index from contigs.fasta.cs.4.1.bif.

Thanks

[blu78]

Hi Nils,

Thanks for your reply.
Here is the command I use (on a much smaller set of reads now -- 1 000 000):

bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -i 1 -I 2,3 -A 1 -K 8 -M 512 -w 0 -n 8 > matchF3_10Mb 2> err_10Mb

I get the same results even if I skip secondary indexes and, as you suggest, I use the following command:

bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -i 1 1,2,3 -A 1 -K 8 -M 512 -w 0 -n 8 > matchF3_10Mb 2> err_10Mb

and even if I do not specify the primary indexes with -i (in that case I would use all my 10 indexes instead of just three).

What is kind of wierd is that in this latter case the segmentation fault occurs after searching index 4, while in the first and second case it occurs after searching index 1.

Any clue on what is going on?

Thanks again for your help...
Cheers

[nilshomer]

Hi guys,

I have a problem with BFAST. I am trying to align about 300 Million Solid reads (about 30GB of data) and using multiple indexes. When looking for CAL I unfortunately get a Segmentation Fault that kills the application. At the moment I am testing the software on a 8 cores machine equipped with 32GB of ram running a Linux Debian.
According to the messages sent to the stderr it seems that the software crashes when it is copying (where?) the reads that are not aligned with the primary index in such a way to search them with the secondary indexes.

In the following, an extract of the output messages I get before the software crashes:

Reads processed: 314999637
Cleaning up index.
Searching index file 1/1 (index #1, bin #1) complete...
Found 14927144 matches.
Found matches for 14927144 reads.
Copying unmatched reads for secondary index search.

Thing is I noticed that while executing BFAST is not using too much RAM (about 1.3GB) but it seems to disk cache quite a lot (eating up all the RAM in this way).

Does anyone have any suggestions on what might be going on?
Any help would be appreciated on how to solve this.
Thanks

Could you give the full command you are using? Also, try avoiding the secondary index search and simply use all your indexes in your primary search. This will improve sensitivity and accuracy, and is the recommended mode indicated in the manual.

[blu78]

Hi guys,

I have a problem with BFAST. I am trying to align about 300 Million Solid reads (about 30GB of data) and using multiple indexes. When looking for CAL I unfortunately get a Segmentation Fault that kills the application. At the moment I am testing the software on a 8 cores machine equipped with 32GB of ram running a Linux Debian.
According to the messages sent to the stderr it seems that the software crashes when it is copying (where?) the reads that are not aligned with the primary index in such a way to search them with the secondary indexes.

In the following, an extract of the output messages I get before the software crashes:

Reads processed: 314999637
Cleaning up index.
Searching index file 1/1 (index #1, bin #1) complete...
Found 14927144 matches.
Found matches for 14927144 reads.
Copying unmatched reads for secondary index search.

Thing is I noticed that while executing BFAST is not using too much RAM (about 1.3GB) but it seems to disk cache quite a lot (eating up all the RAM in this way).

Does anyone have any suggestions on what might be going on?
Any help would be appreciated on how to solve this.
Thanks