AllPaths-LG error at FindError module

[akshaya.ramesh]

Hello,

I am a newbie to using AllPaths and I ran into an error while running the RunallPathsLG algorithm.

After preparing the input files, and halfway through running the RunAllPathsLG command I get an error message with the following output:

------------------ FindErrors -> frag_reads_edit.fastb

290282408 total number of original fragment reads
96.2 mean length of original fragment reads in bases
38.8 % gc content of fragment reads
0.2 % of bases pre-corrected
2299242576 estimated genome size in bases
21.0 % genome estimated to be repetitive (at K=25 scale)
9 estimated genome coverage by fragment reads
0.32 estimated standard deviation of sequencing bias (at K=25 scale)
92.7 % of bases confirmed in cycle 0
1.32 % of bases corrected in cycle 0
0.01 % of bases with conflicting corrections in cycle 0
94.8 % of bases confirmed in cycle 1
0.31 % of bases corrected in cycle 1
0.01 % of bases with conflicting corrections in cycle 1

------------------ CleanCorrectedReads -> frag_reads_corr.25mer.kspec

8.1 % of reads removed because of low frequency kmers

------------------ FillFragments -> filled_reads.fastb

58.1 % of fragment pairs that were filled

------------------ Memory and CPU usage

64 available cpus
126.1 GB of total available memory
389.0 GB of available disk space
50.87 hours of total elapsed time
46.76 hours of total per-module elapsed time
596678.04 hours of total per-module user time
12759.45 effective parallelization factor
115.56 GB memory usage peak

Run directory: /archive/REFERENCE/DATA/RUN
Log directory: /archive/REFERENCE/make_log/DATA/RUN/SUBDIR/2013-05-10T07:55:31

*** Make encountered an error, see above for error messages. ***

I am not really sure as to where I would look for any error's. Since the makefile is generated before the actual process, I do not really understand where the error is. Has anyone encountered something like this before/does anyone have any suggestions?

I did post the same question in the allpaths-user forum, but I havent got any reply yet.

Any insights would be greatly appreciated.

Thanks in advance for your time.
Best,
Akshaya

[Lida]

Hello,

I am a newbie to using AllPaths and I ran into an error while running the RunallPathsLG algorithm.

After preparing the input files, and halfway through running the RunAllPathsLG command I get an error message with the following output:

------------------ FindErrors -> frag_reads_edit.fastb

290282408 total number of original fragment reads
96.2 mean length of original fragment reads in bases
38.8 % gc content of fragment reads
0.2 % of bases pre-corrected
2299242576 estimated genome size in bases
21.0 % genome estimated to be repetitive (at K=25 scale)
9 estimated genome coverage by fragment reads
0.32 estimated standard deviation of sequencing bias (at K=25 scale)
92.7 % of bases confirmed in cycle 0
1.32 % of bases corrected in cycle 0
0.01 % of bases with conflicting corrections in cycle 0
94.8 % of bases confirmed in cycle 1
0.31 % of bases corrected in cycle 1
0.01 % of bases with conflicting corrections in cycle 1

------------------ CleanCorrectedReads -> frag_reads_corr.25mer.kspec

8.1 % of reads removed because of low frequency kmers

------------------ FillFragments -> filled_reads.fastb

58.1 % of fragment pairs that were filled

------------------ Memory and CPU usage

64 available cpus
126.1 GB of total available memory
389.0 GB of available disk space
50.87 hours of total elapsed time
46.76 hours of total per-module elapsed time
596678.04 hours of total per-module user time
12759.45 effective parallelization factor
115.56 GB memory usage peak

Run directory: /archive/REFERENCE/DATA/RUN
Log directory: /archive/REFERENCE/make_log/DATA/RUN/SUBDIR/2013-05-10T07:55:31

*** Make encountered an error, see above for error messages. ***

I am not really sure as to where I would look for any error's. Since the makefile is generated before the actual process, I do not really understand where the error is. Has anyone encountered something like this before/does anyone have any suggestions?

I did post the same question in the allpaths-user forum, but I havent got any reply yet.

Any insights would be greatly appreciated.

Thanks in advance for your time.
Best,
Akshaya

Hi, I have run into the same problem. Did you ever find a solution? Any help will be greatly appreciated!

Lida

[GenoMax]

This is a guess but have you looked in this directory to see if there is an error log file(s) available: /archive/REFERENCE/make_log/DATA/RUN/SUBDIR/...

[akshaya.ramesh]

Hi, I have run into the same problem. Did you ever find a solution? Any help will be greatly appreciated!

Lida

Hello!

I simply re-ran AllPaths and set the Threads to 32. It worked the second time. Have you tried re-running All-Paths? Also, I usually run my denovo assembly with a small portion of the data-set, you could try that as well and see if it works. If you still do not get it to work you can post in the AllPaths forum.

https://groups.google.com/a/broadinstitute.org/forum/#!forum/allpaths-user-forum

[Seth]

Rerun

Hi all
I also encountered this problem. I re-submitted the task and now I'm waiting for the result, hopefully it would work.

Seth

[kammoji]

Hey,

Did you get it working? Does setting the number of threads to 32 work? I get something similar with Allpaths-LG:

"
Program FindErrors failed to make target $(RUN)/frag_reads_edit.readtrack
/bin/sh: @echo: not found
make: *** [/home/jkammone/wrk/Miikkulainen/assembly/allpaths/data2/second_allpaths_assembly/frag_reads_edit.fastb] Error 1
make: *** Deleting file `/home/jkammone/wrk/Miikkulainen/assembly/allpaths/data2/second_allpaths_assembly/frag_reads_edit.fastb'

Fri Jan 24 22:43:51 2014 : make process finished.


Looks like the command FindErrors did not complete:
The following files are missing: 1
$(RUN)/frag_reads_edit.fastb
The following files are incomplete: 2
$(RUN)/frag_reads_edit.qualb
$(RUN)/frag_reads_filt.25mer.kspec
Cleaning up all targets of this command...
Moving $(RUN)/frag_reads_edit.qualb to $(RUN)/makeinfo/frag_reads_edit.qualb.incomplete
Moving $(RUN)/frag_reads_filt.25mer.kspec to $(RUN)/makeinfo/frag_reads_filt.25mer.kspec.incomplete

Fri Jan 24 22:43:52 2014: Compiling assembly report.

------------------ FindErrors -> frag_reads_edit.fastb

323559312 total number of original fragment reads

...

"

Regards.

[akshaya.ramesh]

Yes, that actually fixed it. I am not an expert at this, but increasing the numbers of threads fixed the problem for me. Have you had any luck since?

[kammoji]

Hi,

For me, changing the number of threads did not work. Moreover, I have problems with the pipeline described in the manual:

If I run the pipeline like this (very roughly):

1. PrepareAllPathsInputs
2. RunAllPathsLG

I get:

"
ValidateAllPathsInputs PRE=/home/jkammone/wrk/Miikkulainen/assembly/ \
DATA=allpaths/data K=96 K_LONG=640 LONG_JUMPS=False \
LONG_READS=False REFERENCE=False WARN_ONLY=False \
MM=True _MM_INTERVAL=10 _MM_SUMMARY=False \
_MM_OUT=/home/jkammone/wrk/Miikkulainen/assembly/allpat \
hs/data/makeinfo/ValidateAllPathsInputs.report.mm.Valid \
ateAllPathsInputs
--------------------------------------------------------------------------------



Redirecting standard output to the following files:
/home/jkammone/wrk/Miikkulainen/assembly/allpaths/data/ValidateAllPathsInputs.report
/home/jkammone/wrk/Miikkulainen/assembly/allpaths/data/makeinfo/ValidateAllPathsInputs.report.out.ValidateAllPathsInputs
/home/jkammone/wrk/Miikkulainen/assembly/allpaths/make_log/data/second_allpaths_assembly/test/2014-01-31T10:39:42/ValidateAllPathsInputs.out

You should not have specified a value for K_LONG.
You should not have specified a value for LONG_READS.
You should not have specified a value for REFERENCE.

To see the syntax for this command, type "ValidateAllPathsInputs -h".

make: *** [/home/jkammone/wrk/Miikkulainen/assembly/allpaths/data/ValidateAllPathsInputs.report] Error 255
make: *** Deleting file `/home/jkammone/wrk/Miikkulainen/assembly/allpaths/data/ValidateAllPathsInputs.report'

"

----


In case I run the pipeline like this:

1. PrepareAllPathsInputs
2. ValidateAllPathsInputs #manual validation
3. RunAllPathsLG (set VALIDATE_INPUTS=False)

I get the above error message. The troublesome lines are:

"
Program FindErrors failed to make target $(RUN)/frag_reads_edit.readtrack
/bin/sh: @echo: not found
"

I have no clue why it goes like this. AllPathsLG-forum didn't help either. I am sending email to David Jaffe to Broad to investigate.

Thanks anyway and regards.

[xiaopangding1988]

Hi,
I have the same problem in using Allpath-LG like:
make: *** [/media/2TB/sweetpotato.genome/data/run/makeinfo/frag_reads_edit.fastb] Error 1
Call to new failed, memory usage before call = 214504216k.

Here is the output of top:

Aborting.

I have no idea which part is wrong. Any insights would be greatly appreciated.

Thanks in advance for your time.