[madhubioinfo]

allpaths is exiting in the que showing "E" in log file no error messages also

my pbs script
#!/bin/bash
#PBS -l walltime=48:00:00
#PBS -N 268_allpaths
#PBS -q workq
#PBS -l select=40:ncpus=16:mpiprocs=16
#PBS -l place=scatter:excl
#PBS -V

# comment begins with # followed by space......[IMPORTANT]
# Go to the directory from which you submitted the job
# cd $PBS_O_WORKDIR

module load all_paths-2.2
# path of all_paths
# path : /app/allpathslg
module load openmpi-1.6.4


# ulimit (stack) is needed by the allpaths program
ulimit -s 100000

# prepare data for allpaths:
PrepareAllPathsInput\
DATA_DIR=$PWD/scratch/268_allpaths\
PLOIDY=1\
IN_GROUPS_CSV=/scratch/268_allpaths/in_groups.csv\
IN_LIBS_CSV=/scratch/268_allpaths/in_libs.csv\
OVERWRITE=True\
| tee prepare.out

# Assemble data:
allpathslg\
PRE=$PWD\
DATA_SUBDIR=data\
RUN=run\
SUBDIR=test\
OVERWRITE=True\
| tee -a assemble.out


my csv files

in_groups.csv

file_name, library_name, group_name
/scratch/268_allpaths/SO_2511_268_R1.fastq, illumina, frags
/scratch/268_allpaths/SO_2511_268_R2.fastq, illumina, frags
/scratch/268_allpaths/SO_2511_268_R1.fastq.gz, illumina_short, jumping
/scratch/268_allpaths/SO_2511_268_R2.fastq.gz, illumina_short, jumping

in_libs.csv
library_name, project_name, organism_name, type, paired, frag_size, insert_size, read_orientation, genomic_start, genomic_end
illumina, test assembly, test, fragment, 1, 300bp, 480bp, inward, 0, 0
illumina, test assembly, test, fragment, 1, 300bp, 480bp, inward, 0, 0
illumina, test assembly, test, jumping, 1, , 2k, outward, 0, 0
illumina, test assembly, test, jumping, 1, , 2k, outward, 0, 0

[rdlady]

Hi
at the end it seems that nobody is able to run ALLPATHS is that true?

F.

I'm running it with these options:
RunAllPathsLG PRE=<my dir. REFERENCE_NAME=refs DATA_SUBDIR=data RUN=output OVERWRITE=TRUE USE_LONG_JUMPS=False REFERENCE_FASTA=refs/reference.fasta

And I get this error:




Error: file /scratch/hpc/raquel/allpaths/refs/data/frag_reads_orig.fastb is supposed to already exist, but doesn't.
ForceAssert(IsRegularFile( *it )) at system/MiscUtil.cc:996 failed in function
int MakeMgr::RunMake(int)


Mon Apr 29 11:24:19 2013. Abort. Stopping.

Generating a backtrace...

Dump of stack:

0. CRD::exit(int), in Exit.cc:49
1. yes, in Assert.h:52
2. main, in RunAllPathsLG.cc:3134

[erhuangzi]

I won't make any claim about allpaths-lg being universally runnable, but I can say that we have been using it quite successfully on microbial and fungal projects.

To get workable binaries you will need gcc-4.3.2 or newer and boost-1.38 or newer (building these can be a bear). I can post configure commands if that's useful to anyone.

Finally, I strongly encourage anyone testing allpaths-lg to download the test data supplied by the Broad (ftp://ftp.broadinstitute.org/pub/crd....genome.tar.gz) and get this working before running your own data.

I also have this question, and my gcc is gcc-4.5.2. when I run allpaths-lg using the test data;
/bin/sh: PrepareAllPathsInput: not found
make：********* Error 127
can you help me？ I want to run this software.
thans

[tonybolger]

I tried using the velvet formula for my RAM calculation and about 500 Gb is the answer for human sized genomes. Is that your experience too?

Memory consumption is massively impacted by read quality - trimming is strongly recommended to reduce the pain.

Aslo, how many days approx. would it take? The HPC environment here has a max walltime of 96hrs.

Depends a lot on the hardware - for me, SOAP takes about 2 days, but it can be split into separate steps. I don't have experience of velvet with full-scale genomes, but i suspect you'll go well over that time.

Another question is regarding trimming, did you use any specific software or wrote your own scripts?

I wrote this bad boy to do what i needed.

[vyellapa]

Firstly, I did not use human sized genome. I was doing metagenomics assembly of three known bacterial genome. Secondly, my computing time varied as I was not only the one using the HPC. If you are asking about the quality trimming of sequence reads (illumina or 454 pyrosequence), I see some people suggest seqtrim, prinseq and clean_reads for trimming. I did not do any quality trimming during my comparative study. I have also developed my own script for quality trimming. I compared my script with prinseq and clean_reads which is performing better than them on the overall. Sorry I can give the script now as I am going to publish it. After the publication I will notify soon.

Thank you Himalaya!

[Himalaya]

I tried using the velvet formula for my RAM calculation and about 500 Gb is the answer for human sized genomes. Is that your experience too? Aslo, how many days approx. would it take? The HPC environment here has a max walltime of 96hrs.

Another question is regarding trimming, did you use any specific software or wrote your own scripts?

Firstly, I did not use human sized genome. I was doing metagenomics assembly of three known bacterial genome. Secondly, my computing time varied as I was not only the one using the HPC. If you are asking about the quality trimming of sequence reads (illumina or 454 pyrosequence), I see some people suggest seqtrim, prinseq and clean_reads for trimming. I did not do any quality trimming during my comparative study. I have also developed my own script for quality trimming. I compared my script with prinseq and clean_reads which is performing better than them on the overall. Sorry I can give the script now as I am going to publish it. After the publication I will notify soon.

[[email protected]]

Abyss vs Velvet

I tried using the velvet formula for my RAM calculation and about 500 Gb is the answer for human sized genomes. Is that your experience too? Aslo, how many days approx. would it take? The HPC environment here has a max walltime of 96hrs.

Another question is regarding trimming, did you use any specific software or wrote your own scripts?

[Himalaya]

We have human paired-end data from complete genomics. The read length is 70bp.Which is am appropriate software tool to use? I have come accross, SOPA denovo, allpaths-lg, Velvet and Abyss. Which of these is suitable for use on human data if a HPC environment is available?

I did a small comparative study of soap denovo, velvet, ABYSS and cortex. I could not use allpaths. SOAP devono is fast than other but uses very huge memory. ABYSS does better assembly but velvet is not bad as well. If I were you, I would use ABYSS and velvet both.

[[email protected]]

Which De novo assembler?

We have human paired-end data from complete genomics. The read length is 70bp.Which is am appropriate software tool to use? I have come accross, SOPA denovo, allpaths-lg, Velvet and Abyss. Which of these is suitable for use on human data if a HPC environment is available?

[fishinabarrel]

Hi all,

Finally, allpaths-lg works on my machine.

My OS is CentOS 5.3 and the version of allpaths-ls is 38003.

I would like to recommend retrying allpaths-lg as it's getting familiar.

Thank you Corthay !
I have not retried this tool since about version 36861.

I will give it a go again and see how far I get.

[corthay]

Hi all,

Finally, allpaths-lg works on my machine.

My OS is CentOS 5.3 and the version of allpaths-lg is 38003.

I would like to recommend retrying allpaths-lg as it's getting familiar.

[acopeland]

I won't make any claim about allpaths-lg being universally runnable, but I can say that we have been using it quite successfully on microbial and fungal projects.

To get workable binaries you will need gcc-4.3.2 or newer and boost-1.38 or newer (building these can be a bear). I can post configure commands if that's useful to anyone.

Finally, I strongly encourage anyone testing allpaths-lg to download the test data supplied by the Broad (ftp://ftp.broadinstitute.org/pub/crd....genome.tar.gz) and get this working before running your own data.

[fishinabarrel]

I have been trying Allpaths-LG for a few days now. I got it to compile after dealing with an memory stack trace error (needed to set limit to 100000, rather than the setting of unlimited).

The programs run, but they fail to actually succeed. I get crashes at various stages of their perl scripts. I am posting this just as a word of caution to anyone else out there who might be thinking of using ALLPATHS-LG. Don't waste your time on this until there is some additional evidence that their software can be used universally.

[pengchy]

"may be nobody is able to run ALLPATHS.."
Really?
We also have this question. Could anybody can help us?

[sundarsukumar]

Getting error in allpaths2.2

HI all, Please help me on this.

I am running allpaths 2.2 , actually i was trying 2 install allpaths 3.2 .,while i was trying to install, i couldn't able to install boost library proprely..

Here is my error

sundar@sundar-desktop:/media/ubuntu/Allpaths/allpaths-2.2/src$ ulimit -s 100000

sundar@sundar-desktop:/media/ubuntu/Allpaths/allpaths-2.2/src$ ./RunAllPaths PRE=input/ DATA_SUBDIR=MyTestData RUN=MyRun REFERENCE_NAME=Ecoli TARGETS=import SOURCE_DIR=input/input_source REFERENCE_FASTB=input/input.reference/genome.fastb K=20 EVALUATION=REFERENCE MAXPAR=1 PARALLEL_BATCHES=1

--------------------------------------------------------------------------------
Mon Aug 02 10:26:27 2010 run (pid=4467)
RunAllPaths PRE=input/ DATA_SUBDIR=MyTestData RUN=MyRun REFERENCE_NAME=Ecoli \
TARGETS=import SOURCE_DIR=input/input_source \
REFERENCE_FASTB=input/input.reference/genome.fastb K=20 \
EVALUATION=REFERENCE MAXPAR=1 PARALLEL_BATCHES=1
--------------------------------------------------------------------------------

running on sundar-desktop

MARK_TRUSTED_A_MIN_SUPPORT = 4, MARK_TRUSTED_A_MAX_UNSUPPORTED_TO_TEST = 5, MARK_TRUSTED_B_MIN_SUPPORT = 4, MARK_TRUSTED_B_MAX_UNSUPPORTED_TO_TEST = 5
run dir is
input//Ecoli/MyTestData/MyRun

Found existing genome.fastb file in reference directory.
Ignoring REFERENCE_FASTB option.
targetsToMake = 1
input//Ecoli/MyTestData/reads_orig.fastb

forceTargets = 0



Copy of makefile in: input//Ecoli/make_mgr/MyTestData/MyRun/test/2010-08-02T10:26:27.make
[PAPI]
[PAPI] === Getting data from the source directory into ALLPATHS format ===
[PAPI]
[PAPI] Calling PrepareAllPathsInput to create 11 file(s):
[PAPI]
[PAPI] $(DATA)/ploidy
[PAPI] $(DATA)/reads_orig.fastb
[PAPI] $(DATA)/reads_orig.lane_index
[PAPI] $(DATA)/reads_orig.lane_infos
[PAPI] $(DATA)/reads_orig.lanes
[PAPI] $(DATA)/reads_orig.lengths
[PAPI] $(DATA)/reads_orig.pairto
[PAPI] $(DATA)/reads_orig.pairto_index
[PAPI] $(DATA)/reads_orig.pairtob
[PAPI] $(DATA)/reads_orig.props
[PAPI] $(DATA)/reads_orig.qualb
[PAPI]
[PAPI] from 0 file(s):
[PAPI]
[PAPI]
[PAPI] PrepareAllPathsInput SOURCE_DIR=$(SOURCE) DATA_DIR=$(DATA) READS_OUT="reads_orig" STATS_IN=expected_lib_stats
/bin/sh: PrepareAllPathsInput: not found
make: *** [input//Ecoli/MyTestData/ploidy] Error 127


Mon Aug 02 10:26:28 2010: make finished, run dir was
input//Ecoli/MyTestData/MyRun


*** Make encountered an error, see above for error messages. ***