Myrna stops in paired bowtie alignment

[Serena Rhie]

Hi all,

I am running myrna_local on a paired read solexa+64 data(FASTQ formatted).

I've passed the preprocess step with --pre-filemax 5000 and want to proceed with the following command and given args:

$MYRNA_HOME/myrna_local --reference /home/Serena/myrna-1.0.9/reftools/human_ensembl_58 --input /media/SAMSUNG/Data/myrna/paired_labeled_manifest/preprocessed --output /media/SAMSUNG/Data/myrna/paired_labeled_manifest/output --quality solexa64 --keep-all --cpus 8 --bowtie-args "-m 1 -n 5 -l 10 -q -I 450 -X 550"

However, I keep get the message that wants to retry in 5 seconds:

Non-zero return (65280) after executing command 'gzip -dc /media/SAMSUNG/Data/myrna/paired_labeled_manifest/preprocessed/s_1_1_sequence_8M.txt_3.out.gz | perl /home/Serena/myrna-1.0.9/Align.pl --bowtie /home/Serena/myrna-1.0.9/bin/linux64/bowtie --discard-reads=0 --index-local=/home/Serena/myrna-1.0.9/reftools/human_ensembl_58/index/human_ensembl_58 --partlen=1000000 --qual=solexa64 --truncate=0 --globals=/tmp/myrna/intermediate/3697/globals --discard-mate=0 --pool-trim-length=0 -- --partition 1000000 --mm -t --hadoopout --startverbose -m 1 -n 5 -l 10 -q -I 450 -X 550 >/tmp/myrna/intermediate/3697/align/map-03741 2>/tmp/myrna/intermediate/3697/align.map.pre/map.err/map-03741'
Retrying in 5 seconds...

Eventually the whole process ends up by giving this message:

Fatal error 1.0.9:M130: Out of retries; aborting...

One of the error message that has been generated by Align.pl is

******
* Aborting master loop because child 3745 failed
* (other children may also have failed)
* Input file or string was:
*   /media/SAMSUNG/Data/myrna/paired_labeled_manifest/preprocessed/s_1_1_sequence_8M.txt_389.out.gz:
* Error message is in file: /tmp/myrna/intermediate/3697/align.map.pre/map.err/map-03745, also printed below
******
* Align.pl: s3cmd: found: /usr/bin/s3cmd, given: 
* Align.pl: jar: found: /usr/java/bin/jar, given: 
* Align.pl: hadoop: found: , given: 
* Align.pl: wget: found: /usr/bin/wget, given: 
* Align.pl: s3cfg: 
* Align.pl: bowtie: found: , given: /home/Serena/myrna-1.0.9/bin/linux64/bowtie
* Align.pl: partition len: 1000000
* Align.pl: ref: 
* Align.pl: quality: solexa64
* Align.pl: truncate at: 0
* Align.pl: discard mate: 0
* Align.pl: discard reads < truncate len: 0
* Align.pl: SAM passthrough: 0
* Align.pl: Straight through: 0
* Align.pl: globals directory: /tmp/myrna/intermediate/3697/globals
* Align.pl: pool replicates?: 0
* Align.pl: pool trim length: 0
* Align.pl: pool technical replicates?: 0
* Align.pl: local index path: /home/Serena/myrna-1.0.9/reftools/human_ensembl_58/index/human_ensembl_58
* Align.pl: bowtie args: --partition 1000000 --mm -t --hadoopout --startverbose -m 1 -n 5 -l 10 -q -I 450 -X 550
* Align.pl: ls -al
* Align.pl: total 6380
* drwxr-xr-x  2 root root    4096 2010-09-16 21:54 .
* drwxr-xr-x 10 root root    4096 2010-09-16 21:53 ..
* -rw-r--r--  1 root root 2165000 2010-09-16 21:53 .tmp.Align.pl.3761.Normal-Subject1-Lane1
* -rw-r--r--  1 root root    5660 2010-09-16 21:53 .tmp.Align.pl.3761.Normal-Subject1-Lane1.err
* -rw-r--r--  1 root root 2165000 2010-09-16 21:53 .tmp.Align.pl.3955.Normal-Subject1-Lane1
* -rw-r--r--  1 root root    5660 2010-09-16 21:53 .tmp.Align.pl.3955.Normal-Subject1-Lane1.err
* -rw-r--r--  1 root root 2165000 2010-09-16 21:54 .tmp.Align.pl.4200.Normal-Subject1-Lane1
* -rw-r--r--  1 root root    5660 2010-09-16 21:54 .tmp.Align.pl.4200.Normal-Subject1-Lane1.err
* Align.pl: Read first line of stdin:
* FN:s_1_1_sequence_8M.txt;LB:Normal-Subject1-Lane1;RN:@GA05_0001:1:9:13860:6569#0/2	CCTAAACACTTTCTTCCTGAGATGTTAAGCAAAGTANTCATCCTGTCACTAGATAGAAGCGATGAAGATAAAGAAAAAGCAAGTTCTTTGATCAGTTTACT	bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb`B````^```bbabbbbbbabbbbabbbbbabbbbbbbab___aab_b\`bbaab^`bbb_ab^bb	TGCAAACTGTGCTAAATAGGATTTCATCAAAGGGATGTCAACCTCCAGTTTGGGACACTGGTCCAATACATTCAGGAAAGCCTGCATGAACTTGTCACTTG	bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbabbb`LNOONbbbbc`
* Align.pl: Read last line of stdin:
* FN:s_1_1_sequence_8M.txt;LB:Normal-Subject1-Lane1;RN:@GA05_0001:1:9:14250:13470#0/2	TAGAAAACAAGTAATTGCTAAATCAAAATAGAATATNGTATACTTTTAATCTTTCATTTTTTTTCTTTCAGGTTTTTCAGTTGCCTTCTTTCCTAGATCAC	bbbbbbbbbaabbbbbbbbbbbbbbbbbbbbbbbc`B`^^`\``^bbbbbbbbbbbbbbbbba_aaa_aabbabbbbbbbabbbb_abbabbabb\bbb^^	CTGAGGTATTCTCCATGCTATAACCCAGTTTAGCTGATGCATTTGGAGCTTGGGTGCTGAAATTAAATATAACCTATTTGAGTTAAGGATTTATTACTAGT	bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbcbbbbbbbabbbbbabbbbabbbbbbbcb_bcbb\babbbbbccbc`bcbb_`b^
* Align.pl: 5000 reads downloaded
* reporter:counter:Bowtie,Reads downloaded,5000
* reporter:counter:Bowtie,Reads (all) passing filters,5000
* reporter:counter:Bowtie,Reads (unpaired) passing filters,0
* reporter:counter:Bowtie,Reads (paired) passing filters,5000
* reporter:counter:Bowtie,Reads skipped due to -discard-reads,0
* reporter:counter:Bowtie,Reads skipped due to -truncate-discard,0
* reporter:counter:Bowtie,Mates skipped due to -discard-mate,0
* reporter:counter:Bowtie,Reads (mates) truncated due to -truncate*,0
* reporter:counter:Bowtie,Reads of length 101 before filtering,10000
* reporter:counter:Bowtie,Occurrences of raw quality value [80:90),31853
* reporter:counter:Bowtie,Occurrences of raw quality value [60:70),115467
* reporter:counter:Bowtie,Occurrences of raw quality value [70:80),12217
* reporter:counter:Bowtie,Occurrences of raw quality value [90:100),850463
* reporter:counter:Bowtie,Occurrences of phred-33 quality value [10:20),17906
* reporter:counter:Bowtie,Occurrences of phred-33 quality value [0:10),116557
* reporter:counter:Bowtie,Occurrences of phred-33 quality value [30:40),804186
* reporter:counter:Bowtie,Occurrences of phred-33 quality value [20:30),71351
* Align.pl: 5000 reads downloaded
* Align.pl: head -4 .tmp.Align.pl.4432.Normal-Subject1-Lane1:
* Align.pl: LB:Normal-Subject1-Lane1	CCTAAACACTTTCTTCCTGAGATGTTAAGCAAAGTANTCATCCTGTCACTAGATAGAAGCGATGAAGATAAAGAAAAAGCAAGTTCTTTGATCAGTTTACT	CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCA%AAAA?AAACCBCCCCCCBCCCCBCCCCCBCCCCCCCBC@@@BBC@C=ACCBBC?ACCC@BC?CC	TGCAAACTGTGCTAAATAGGATTTCATCAAAGGGATGTCAACCTCCAGTTTGGGACACTGGTCCAATACATTCAGGAAAGCCTGCATGAACTTGTCACTTG	CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCBCCCA-/00/CCCCDA
* LB:Normal-Subject1-Lane1	CAGAAGCACAAGAGCATCGAGGAGATCGTGCGGCGANTTGACCCCAACAAGTACCCTGTGCCAGAAAATTGGCTCCACAAGGAGGCTCACCAGCTCTTCTT	CCCCCCCCCCCCCCCCCCCCCCCCCCCCBCCCCCCA%AABCCCCCCCCCCCACCCCBC>CCDBCD<B@B<BBDAAB>B>B>B6D?BAAABCAABA>D<?D6	CTGCCTTGGCGGCTCTTACTCAGCCTCTTGACCCCACTGCGGATTCGCTCCTCAGAGAACTGCTTTTCACCACACATGAACTTGATCAGCTCTTCTTCATT	CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCBDDCCCDCCDCBBBC@DD@DCDDCDBBDCDDBA>BB8@CBBBABCADB>>:BBBBD?8A
* LB:Normal-Subject1-Lane1	GGGCGAGCGCCGTTCCGAAGGGACGGGCGATGGCCTNCGTTGCCCTCGGCCGATCGAAAGGGAGTCGGGGTCAGATCCCCGAATCCGGAGGGGCGGGGGGG	CCCCCCCCCCCCCCCCCCCCCCCCCCCCCBADDDC;%?@79;5<:CDDDAB@:BBB>B>>@@2@1>>@@%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%	CGGGGCTCCCGCCGGCTTCTCCGGGATCGGTCGCGTTACCGCACTGGCCGCCTCGCGGCGCCCATCTCCGCCACTCCGGATTCGGGGATCTGAACCCGACT	CCCCCCCCCCCCCCCCCCCCCCACC?ACBC@CCAC@CBCCDCCCB=B;@B<A8><92<<9=@@8>>5>A>BBD?:>@3A899;>A??+@@%%%%%%%%%%%
* LB:Normal-Subject1-Lane1	CGTGGGTTTAAGTCCCATTGGTCTAGTAAGGGCTTANCTTAATTAAAGTGGCTGATTTGCGTTCAGTTGATGCCGAGGGGGGTTTGTTAAAGTTGTTAAGA	CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCA%AAABABBBCCC@CCCCC457;43@%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%	CGGGCGTAAGCAATCTGACAAATGATATCTCTGTTTGTCACACGAACTATCATCCTGTATTTGGGTGTGTTGTATTTATTTTTATCTTGTATCACCAAGCG	CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCDCDCCCCCBCCCCCCBBB@B@CCCCCCC@CCA;CCA
* Align.pl: tail -4 .tmp.Align.pl.4432.Normal-Subject1-Lane1:
* Align.pl: LB:Normal-Subject1-Lane1	GCTGGGAGGCAGAGGTTGCAGTGAGCCAAAATTGTGNCACTGCACTGCAGCCTNNNNAACAGAGTANNNNNNNNNNNNNNNNNCAACAACAACAACAAAAN	CCCCCC@CCCCCCCC>CCCCC=A=C7B?;ABCBCB<%A<?>?>;;ADBBD%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%	TNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN	%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
* LB:Normal-Subject1-Lane1	CCCCAGGAGCCTTGCCTCAGAGACCAACTGAGTGGCNGGGTTCTCCCCAAGCAGCCCCCAGCACCCCCCCCCCCCCCCCCCCGCCCCCCCCGCACCGCCCC	CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCDCBCCA%A?%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%	GGGGGGTCCCCAAGCCCAGGGTGGACGAGGAAAAGGTCAGGAAATAGAGGATTGTCCTGAGCCCTCCTGGCCATGGGGGCCGACCCAGTGGGCACTGAGGC	CCCCCCCCCCCCCCCCCCCCCACCCCCBCCCCCCCDBCCCCBC@CCCDCCDDCC=BCBBBCCCCBDCBBBBB4BB?@>=9CBDCB=9=.21187+==>A@B
* LB:Normal-Subject1-Lane1	GATACACTTGGACAGGACCAAGAGGCATTTCACTGCNATGAAACAAGGCAGGAAGGGATTCTAATACACACACCAGGAAGCACTCCTGCCCCTCAGAGGTC	000//=AA<=:A=;AC@B@BCC@ABC@@CC@C@4@0%00000.0/BC><?;82983-,78A?<?@??8997;;--?1115<5<<<<.;A%%%%%%%%%%%%	CTTTTATTAAAAAAGTATAGTATATATAATACAAAACAATAACCCTTCTGGGGTTTCTTGTGGCGGGTGGGATAGTAGCACACGCGGGCCTCCCAAACACA	CACCC@C>=@CC?CC?CCCCCCCCCCCCCCCCCC4CCC:>CCCCCCCCCCC@C>CCCCCCACCCC%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
* LB:Normal-Subject1-Lane1	TAGAAAACAAGTAATTGCTAAATCAAAATAGAATATNGTATACTTTTAATCTTTCATTTTTTTTCTTTCAGGTTTTTCAGTTGCCTTCTTTCCTAGATCAC	CCCCCCCCCBBCCCCCCCCCCCCCCCCCCCCCCCDA%A??A=AA?CCCCCCCCCCCCCCCCCB@BBB@BBCCBCCCCCCCBCCCC@BCCBCCBCC=CCC??	CTGAGGTATTCTCCATGCTATAACCCAGTTTAGCTGATGCATTTGGAGCTTGGGTGCTGAAATTAAATATAACCTATTTGAGTTAAGGATTTATTACTAGT	CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCDCCCCCCCBCCCCCBCCCCBCCCCCCCDC@CDCC=CBCCCCCDDCDACDCC@AC?
* Align.pl:   ...ensuring reference jar is installed first
* Align.pl: Running: /home/Serena/myrna-1.0.9/bin/linux64/bowtie --partition 1000000 --mm -t --hadoopout --startverbose -m 1 -n 5 -l 10 -q -I 450 -X 550 --12 .tmp.Align.pl.4432.Normal-Subject1-Lane1 /home/Serena/myrna-1.0.9/reftools/human_ensembl_58/index/human_ensembl_58 2>.tmp.Align.pl.4432.Normal-Subject1-Lane1.err
* cp * /tmp/error/Align.pl.4432/
* cp: cannot stat `*': No such file or directory
* cp .* /tmp/error/Align.pl.4432/
* cp: omitting directory `.'
* cp: omitting directory `..'
* /home/Serena/myrna-1.0.9/bin/linux64/bowtie exited with non-zero exitlevel 256; see /tmp/error/Align.pl.4432 on serena-fedora
******

BTW, the read size is up to 120 bp long.

Thank you in advance,
Serena Rhie

[Ben Langmead]

* Align.pl: Running: /home/Serena/myrna-1.0.9/bin/linux64/bowtie --partition 1000000 --mm -t --hadoopout --startverbose -m 1 -n 5 -l 10 -q -I 450 -X 550 --12 .tmp.Align.pl.4432.Normal-Subject1-Lane1

Hi Serena,

That the error message is confusing is my fault - I'm still trying to make the propagation of error messages cleaner and future versions will do this better. In this case, the error message being emitted by Bowtie wasn't being shown to you when it should have.

The root problem seems to be that you specified -n 5 : the -n parameter can be set to a maximum of 3.

Thanks,
Ben

[Serena Rhie]

Thanks for the fast reply! Wish this is the last question!

What a quick response!
Thank you Ben, seems like I've passed the processing input stage with option -n 3.

However, I'm getting another error message:

(...obmitting)
Pid 8705 processing input /tmp/myrna/intermediate/5199/olaps/part-08511 [30 of 32]...
Pid 8706 processing input /tmp/myrna/intermediate/5199/olaps/part-08572 [31 of 32]...
Pid 8707 processing input /tmp/myrna/intermediate/5199/olaps/part-08590 [32 of 32]...
Factored 32 files into 2 non-empty tasks
Pid 8708 sorting task task-00000 [1 of 2]...
Pid 8709 sorting task task-00001 [2 of 2]...
Pid 8709 reducing task task-00001 [2 of 2]...
Pid 8708 reducing task task-00000 [1 of 2]...
******
* Aborting master loop because child 8709 failed
* (other children may also have failed)
* Input file or string was:
*   /tmp/myrna/intermediate/5199/normal.reduce.pre/reduce.stasks/stask-08709
* Error message is in file: /tmp/myrna/intermediate/5199/normal.reduce.pre/reduce.err/epart-08709, also printed below
******
* Normalization: lup
* Output URL:    /tmp/myrna/intermediate/5199/count
* Processing label Normal-Subject1-Lane1/2
* stat /tmp/myrna/intermediate/5199/count/Normal-Subject1-Lane1/2.txt >& /dev/null
* stat /tmp/myrna/intermediate/5199/count/Normal-Subject1-Lane1/2.norm >& /dev/null
* stat /tmp/myrna/intermediate/5199/count/Normal-Subject1-Lane1/2.norms >& /dev/null
* mv: cannot move `.tmp.Normal.pl.8711' to `Normal-Subject1-Lane1/2.txt': No such file or directory
* stat /tmp/myrna/intermediate/5199/count/2.txt >& /dev/null
* cp Normal-Subject1-Lane1/2.txt /tmp/myrna/intermediate/5199/count/ >&2
* cp: cannot stat `Normal-Subject1-Lane1/2.txt': No such file or directory
* Can't load Normal-Subject1-Lane1/2.txt to /tmp/myrna/intermediate/5199/count/ (256)
******
Fatal error 1.0.9:R330: Aborting because child with PID 8709 exited abnormally

When requesting support, please include the full output printed here.
If a child process was the cause of the error, the output should
include the relevant error message from the child's error log.  You may
be asked to provide additional files as well.
Non-zero exitlevel from Normal stage

Could you give me some clues ?

Thanks,
Serena Rhie

[Ben Langmead]

Hi Serena,

Is this still a problem? I saw your other post which seemed to indicate that you had got further. If it's still a problem, I'll take a look soon.

Ben

[Serena Rhie]

Seems like I'm facing the same problem again.
(I haven't noticed that these were the same!)
Sorry for dup posts.
Let's continue on this thread: http://seqanswers.com/forums/showthread.php?p=25573

Thanks,
Serena