Tweet
I recently got access to a very powerful machine for the purpose of de novo sequencing.
I am using Ray-v2.0.0-rc8
Unfortunately, I have been unable to get ray working on even the smallest test cases with 3 different compilers (Intel, GCC, some other compiler that comes with the system).
Most of my runs end like this:
With input scripts such as
Similar output when trying smaller ecoli file with an intimidating number of PEs. I tried a similar simulation with only 64 PEs but the failure was the same.
(gave neither error nor yield, was build without debug symbols)
1. I am trying to figure out how much ram I need per PE
2. Does anybody have a minimal input output example
3. I haven't done much de-novo assembly and was wondering if there are better programs for eukaryote genome assembly
4. Has anybody tried SRR034, sequences SRR034939-34975?
I am using Ray-v2.0.0-rc8
Unfortunately, I have been unable to get ray working on even the smallest test cases with 3 different compilers (Intel, GCC, some other compiler that comes with the system).
Most of my runs end like this:
Rank 49: assembler memory usage: 3813964 KiB
Rank 75 reached 400 vertices from seed 91, flow 1
Speed RAY_SLAVE_MODE_EXTENSION 2141 units/second
Rank 75: assembler memory usage: 3816056 KiB
Stack walkback for Rank 0 starting:
[email protected]:113
[email protected]
[email protected]
Machine::start()@0x40746f
ComputeCore::runVanilla()@0x500957
MessageProcessor::call_RAY_MPI_TAG_ASK_IS_ASSEMBLED(Message*)@0x45ec8e
Vertex::isAssembled()@0x4db3a0
Stack walkback for Rank 0 done
Process died with signal 11: 'Segmentation fault'
Forcing core dumps of ranks 0, 5, 36, 64, 8, 29, 56, 10, 12, 109, 13, 52, 66, 110
View application merged backtrace tree file with: statview atpMergedBT.dot
_pmiu_daemon(SIGCHLD): [NID 00736] [c1-0c0s0n2] [Sun Jun 24 06:03:13 2012] PE RANK 1 exit signal Killed
_pmiu_daemon(SIGCHLD): [NID 00767] [c1-0c0s0n1] [Sun Jun 24 06:03:13 2012] PE RANK 98 exit signal Killed
[NID 00736] 2012-06-24 06:03:13 Apid 6339046: initiated application termination
Application 6339046 exit codes: 137
Application 6339046 exit signals: Killed
Application 6339046 resources: utime ~39431s, stime ~74s
Rank 75 reached 400 vertices from seed 91, flow 1
Speed RAY_SLAVE_MODE_EXTENSION 2141 units/second
Rank 75: assembler memory usage: 3816056 KiB
Stack walkback for Rank 0 starting:
[email protected]:113
[email protected]
[email protected]
Machine::start()@0x40746f
ComputeCore::runVanilla()@0x500957
MessageProcessor::call_RAY_MPI_TAG_ASK_IS_ASSEMBLED(Message*)@0x45ec8e
Vertex::isAssembled()@0x4db3a0
Stack walkback for Rank 0 done
Process died with signal 11: 'Segmentation fault'
Forcing core dumps of ranks 0, 5, 36, 64, 8, 29, 56, 10, 12, 109, 13, 52, 66, 110
View application merged backtrace tree file with: statview atpMergedBT.dot
_pmiu_daemon(SIGCHLD): [NID 00736] [c1-0c0s0n2] [Sun Jun 24 06:03:13 2012] PE RANK 1 exit signal Killed
_pmiu_daemon(SIGCHLD): [NID 00767] [c1-0c0s0n1] [Sun Jun 24 06:03:13 2012] PE RANK 98 exit signal Killed
[NID 00736] 2012-06-24 06:03:13 Apid 6339046: initiated application termination
Application 6339046 exit codes: 137
Application 6339046 exit signals: Killed
Application 6339046 resources: utime ~39431s, stime ~74s
num="124"
aprun -n $124 ./fancierRayDEBUG/Ray -o Assembly$num -k 31 \
-p \
Sample$num/ERR011117_1.fastq.gz \
Sample$num/ERR011117_2.fastq.gz \
-p \
Sample$num/ERR011118_1.fastq.gz \
Sample$num/ERR011118_2.fastq.gz \
-p \
Sample$num/ERR011119_1.fastq.gz \
Sample$num/ERR011119_2.fastq.gz \
-p \
Sample$num/ERR011120_1.fastq.gz \
Sample$num/ERR011120_2.fastq.gz \
-p \
Sample$num/ERR011121_1.fastq.gz \
Sample$num/ERR011121_2.fastq.gz \
-p \
Sample$num/ERR011122_1.fastq.gz \
Sample$num/ERR011122_2.fastq.gz \
-p \
Sample$num/ERR011123_1.fastq.gz \
Sample$num/ERR011123_2.fastq.gz >& myOutput$num.out
aprun -n $124 ./fancierRayDEBUG/Ray -o Assembly$num -k 31 \
-p \
Sample$num/ERR011117_1.fastq.gz \
Sample$num/ERR011117_2.fastq.gz \
-p \
Sample$num/ERR011118_1.fastq.gz \
Sample$num/ERR011118_2.fastq.gz \
-p \
Sample$num/ERR011119_1.fastq.gz \
Sample$num/ERR011119_2.fastq.gz \
-p \
Sample$num/ERR011120_1.fastq.gz \
Sample$num/ERR011120_2.fastq.gz \
-p \
Sample$num/ERR011121_1.fastq.gz \
Sample$num/ERR011121_2.fastq.gz \
-p \
Sample$num/ERR011122_1.fastq.gz \
Sample$num/ERR011122_2.fastq.gz \
-p \
Sample$num/ERR011123_1.fastq.gz \
Sample$num/ERR011123_2.fastq.gz >& myOutput$num.out
Rank 1: assembler memory usage: 3331872 KiB
Rank 1091: assembler memory usage: 3330848 KiB
Rank 1135: assembler memory usage: 3330848 KiB
Application 6339025 resources: utime ~68335618s, stime ~49747s
Rank 1091: assembler memory usage: 3330848 KiB
Rank 1135: assembler memory usage: 3330848 KiB
Application 6339025 resources: utime ~68335618s, stime ~49747s
aprun -n4096 -N16 -d2 ./fancierRay/Ray --show-memory-usage -o secoliAssembly$num -k 23 \
-p secoliSample$num\SRR001665_1.fastq.gz \
secoliSample$num\SRR001665_2.fastq.gz \
-p secoliSample$num\SRR001666_1.fastq.gz \
secoliSampel$num\SRR001666_2.fastq.gz >& secolimyOutput$num.out
-p secoliSample$num\SRR001665_1.fastq.gz \
secoliSample$num\SRR001665_2.fastq.gz \
-p secoliSample$num\SRR001666_1.fastq.gz \
secoliSampel$num\SRR001666_2.fastq.gz >& secolimyOutput$num.out
2. Does anybody have a minimal input output example
3. I haven't done much de-novo assembly and was wondering if there are better programs for eukaryote genome assembly
4. Has anybody tried SRR034, sequences SRR034939-34975?
Comment