[Gus]

So I know this is pretty late and I am not sure if this is THE source of your problem however it warrants pointing out:

it looks like you tried to use a bowtie 1 index with tophat 2 without telling tophat that you are giving it the old type of index. If you want to do this you must pass tophat the "--bowtie1" switch so that it knows how to handle the indexes. Bowtie 2 indexes are not interchangeable with bowtie 1.

Hello, This is my first post.
i am using Tophat 2.0.6 with Bowtie 0.12 and i am trying to run this command on my clusters:
Quote:
tophat -o /fastspace/bioinfo_projects/asfaw_degu/th2_CS21_2 -i 10 -I 11000 --min-coverage-intron 10 --max-coverage-intron 11000 --min-segment-intron 10 --max-segment-intron 11000 -p 22 -G /storage16/projects/asfaw_degu/02.genome/Vitis_vinifera.IGGP_12x.15.gtf -M /storage16/projects/asfaw_degu/02.genome/Vitis_vinifera /storage16/projects/asfaw_degu/01.fastq/00.Project_Aaron_Fait/Sample_CS_21/R1/CS_21_TAGCTT_L006_R1.fastq /storage16/projects/asfaw_degu/01.fastq/00.Project_Aaron_Fait/Sample_CS_21/R2/CS_21_TAGCTT_L006_R2.fastq

[Gus]

I am still running into problems at this stage even with 2.0.6. Some of my experiments exit fine and other read sets fail here, sometimes producing a "unmapped.bam" file and sometimes not (even with the same input). Its not always the same error either. Its always an error in "reports" but the args afterward are not always the same.

Here is my last stderr output. The python traceback at the top is just because I have written a python script to wrap the whole process and automate sending results to the next step.

Traceback (most recent call last):
File "/home/gus/Dropbox/common/rnaseq_cmds/tophat2cufflinks.py", line 300, in main
stdout,stderr,cmd_string,tophat_out_dir = tophat_phase(i,yargs) # tophat call
File "/home/gus/Dropbox/common/rnaseq_cmds/tophat2cufflinks.py", line 148, in tophat_phase
stdout,stderr = runExternalApp(progName='tophat',argStr=argStr)
File "/home/gus/lib/python/dist-packages/gfunc/externals.py", line 57, in runExternalApp
raise SystemCallError(process.returncode,result[1],progName)
SystemCallError: ERROR in tophat:
[2013-02-06 23:43:01] Beginning TopHat run (v2.0.6)
-----------------------------------------------
[2013-02-06 23:43:01] Checking for Bowtie
Bowtie version: 2.0.5.0
[2013-02-06 23:43:01] Checking for Samtools
Samtools version: 0.1.18.0
[2013-02-06 23:43:01] Checking for Bowtie index files
[2013-02-06 23:43:01] Checking for reference FASTA file
[2013-02-06 23:43:01] Generating SAM header for /home/gus/data/bowtie2_indexes/aaegypti.SUPERCONTIGS-Liverpool.AaegL1.renamed
format: fastq
quality scale: phred33 (default)
[2013-02-06 23:43:02] Reading known junctions from GTF file
[2013-02-06 23:43:03] Preparing reads
left reads: min. length=50, max. length=50, 79545421 kept reads (11140 discarded)
right reads: min. length=50, max. length=50, 79547240 kept reads (9321 discarded)
[2013-02-07 00:12:41] Creating transcriptome data files..
[2013-02-07 00:13:00] Building Bowtie index from aaegypti.BASEFEATURES_Liverpool-AaegL1.3.fa
[2013-02-07 00:13:57] Mapping left_kept_reads to transcriptome aaegypti.BASEFEATURES_Liverpool-AaegL1.3 with Bowtie2
[2013-02-07 00:40:08] Mapping right_kept_reads to transcriptome aaegypti.BASEFEATURES_Liverpool-AaegL1.3 with Bowtie2
[2013-02-07 01:06:26] Resuming TopHat pipeline with unmapped reads
[2013-02-07 01:06:26] Mapping left_kept_reads.m2g_um to genome aaegypti.SUPERCONTIGS-Liverpool.AaegL1.renamed with Bowtie2
[2013-02-07 01:28:43] Mapping left_kept_reads.m2g_um_seg1 to genome aaegypti.SUPERCONTIGS-Liverpool.AaegL1.renamed with Bowtie2 (1/2)
[2013-02-07 01:42:06] Mapping left_kept_reads.m2g_um_seg2 to genome aaegypti.SUPERCONTIGS-Liverpool.AaegL1.renamed with Bowtie2 (2/2)
[2013-02-07 01:55:36] Mapping right_kept_reads.m2g_um to genome aaegypti.SUPERCONTIGS-Liverpool.AaegL1.renamed with Bowtie2
[2013-02-07 02:17:18] Mapping right_kept_reads.m2g_um_seg1 to genome aaegypti.SUPERCONTIGS-Liverpool.AaegL1.renamed with Bowtie2 (1/2)
[2013-02-07 02:31:13] Mapping right_kept_reads.m2g_um_seg2 to genome aaegypti.SUPERCONTIGS-Liverpool.AaegL1.renamed with Bowtie2 (2/2)
[2013-02-07 02:45:07] Searching for junctions via segment mapping
[2013-02-07 02:54:03] Retrieving sequences for splices
[2013-02-07 02:54:59] Indexing splices
[2013-02-07 02:55:26] Mapping left_kept_reads.m2g_um_seg1 to genome segment_juncs with Bowtie2 (1/2)
[2013-02-07 02:57:58] Mapping left_kept_reads.m2g_um_seg2 to genome segment_juncs with Bowtie2 (2/2)
[2013-02-07 03:00:31] Joining segment hits
[2013-02-07 03:02:31] Mapping right_kept_reads.m2g_um_seg1 to genome segment_juncs with Bowtie2 (1/2)
[2013-02-07 03:05:24] Mapping right_kept_reads.m2g_um_seg2 to genome segment_juncs with Bowtie2 (2/2)
[2013-02-07 03:08:10] Joining segment hits
[2013-02-07 03:10:28] Reporting output tracks
[FAILED]
Error running /home/gus/src/tophat-2.0.6.Linux_x86_64/tophat_reports --min-anchor 8 --splice-mismatches 0 --min-report-intron 50 --max-report-intron 500000 --min-isoform-fraction 0.15 --output-dir /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/ --max-multihits 20 --max-seg-multihits 40 --segment-length 25 --segment-mismatches 2 --min-closure-exon 100 --min-closure-intron 50 --max-closure-intron 5000 --min-coverage-intron 50 --max-coverage-intron 20000 --min-segment-intron 50 --max-segment-intron 500000 --read-mismatches 2 --read-gap-length 2 --read-edit-dist 2 --read-realign-edit-dist 3 --max-insertion-length 3 --max-deletion-length 3 -z gzip -p8 --inner-dist-mean 125 --inner-dist-std-dev 25 --gtf-annotations /home/gus/data/vectorbase_data/aaegypti/Geneset/aaegypti.BASEFEATURES_Liverpool-AaegL1.3.gtf --gtf-juncs /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/aaegypti.juncs --no-closure-search --no-coverage-search --no-microexon-search --library-type fr-unstranded --sam-header /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/aaegypti.SUPERCONTIGS-Liverpool.AaegL1.renamed_genome.bwt.samheader.sam --report-discordant-pair-alignments --report-mixed-alignments --samtools=/usr/local/bin/samtools --bowtie2-max-penalty 6 --bowtie2-min-penalty 2 --bowtie2-penalty-for-N 1 --bowtie2-read-gap-open 5 --bowtie2-read-gap-cont 3 --bowtie2-ref-gap-open 5 --bowtie2-ref-gap-cont 3 /home/gus/data/bowtie2_indexes/aaegypti.SUPERCONTIGS-Liverpool.AaegL1.renamed.fa /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/junctions.bed /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/insertions.bed /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/deletions.bed /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/fusions.out /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/accepted_hits /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/left_kept_reads.m2g.bam,/home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/left_kept_reads.m2g_um.mapped.bam,/home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/left_kept_reads.m2g_um.candidates /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/left_kept_reads.bam /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/right_kept_reads.m2g.bam,/home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/right_kept_reads.m2g_um.mapped.bam,/home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/right_kept_reads.m2g_um.candidates /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/right_kept_reads.bam
Loaded 48751 GFF junctions from /home/gus/data/4mosqs_mg_RNAseq/tophat_Aa_4/tmp/aaegypti.juncs..
RETURN_STATE: 1.

[catbus]

Same problem for me, "Reporting output tracks [FAILED]" and no diagnostic information.

This is with Tophat 2.0.9.

I am re-downloading my hg19 fasta files and rebuilding the indexes in an effort to solve this (maybe that will fix it, according to some other seqanswers comments).

[catbus]

I can report that downloading a totally fresh "iGenome" hg19 fasta + gtf annotation + pre-computed bowtie 2 index did not fix the problem.

I assume this is actually a bug in Tophat. I have tried it with many different input files (but all from this one project), so either there is something systematically wrong with all the input fasta files for this project, or there is a bug in Tophat that only occurs in rare conditions.

[2013-08-08 03:38:30] Reporting output tracks
        [FAILED]
Error running /work/Apps/Bio/tophat/tophat-2.0.9/tophat_reports  [...]

[catbus]

SOLVED! (Sort of. Looks like a bug?)

I solved this problem by removing "--no-discordant" from my tophat parameters.

I tried a TON of combinations of parameters, and the "tophat_reports" only gave an error when "--no-discordant" was present.

So I think it's a tophat bug that only occurs under specific conditions and with --no-discordant activated.

This is true for at least Tophat 2.0.8b and Tophat 2.0.9.

[middlemale]

same problem

I faced the same problem with tophat 2.0.9 (bowtie 2.10.0) for my paired-end rna-seq data.

the size of each file is ~3.6GB, 51 bases, my hardware is 12 core x86 machine with 96GB memory and 64_bit debian.

If running tophat2 with --no-discordant, 'reporting output tracks [failed]' happened. it will generate "NORMAL" results without --no-discordant param, but how reliable these results are ?

Next I tried analysing 10,000 first lines of each file with --no-discordant param, it produced "NORMAL" results. odd.

[gs512]

possible working solution

Hi everyone, I've been running in the same error while trying to resume tophat jobs.

So I took a look at the code and found that

the following line was not working :

bowtie_sam_header_filename = tmp_dir + idx_prefix.split('/')[-1]

because of the following error :

AttributeError: 'NoneType' object has no attribute 'split'

meaning that idx_prefix is not considered as a python string, so I edited the line as follow:

bowtie_sam_header_filename = tmp_dir + str(idx_prefix).split('/')[-1]

casting idx_prefix to string

and did the same modification for the line :

bwt_idx_name = bwt_idx_prefix.split('/')[-1]

editing it to

bwt_idx_name = str(bwt_idx_prefix).split('/')[-1]

The job seems to have resumed and is still running I'll give more updates at the end of the run

Let me know if that works for you too

[gs512]

The jobs resumed properly

[Parharn]

gs512 can you explain it for dummies? I have same problem here. Where should I make these changes?

Thanks

[Parharn]

The error I get is:




parham@parham-biolinux[pombe] tophat --no-discordant -p 8 -G genes.gtf -o tag1tag2_thout genome S0001_tag1_ATCACG_L004_R1_001.fastq S0001_tag2_CGATGT_L004_R1_001.fastq

[2014-01-06 17:28:50] Beginning TopHat run (v2.0.9)
-----------------------------------------------
[2014-01-06 17:28:50] Checking for Bowtie
Bowtie 2 not found, checking for older version..
Bowtie version: 1.0.0.0
[2014-01-06 17:28:50] Checking for Samtools
Samtools version: 0.1.19.0
[2014-01-06 17:28:50] Checking for Bowtie index files (genome)..
[2014-01-06 17:28:50] Checking for reference FASTA file
[2014-01-06 17:28:50] Generating SAM header for genome
format: fastq
quality scale: phred33 (default)
[2014-01-06 17:28:50] Reading known junctions from GTF file
[2014-01-06 17:28:51] Preparing reads

WARNING: read pairing issues detected (check prep_reads.log) !

left reads: min. length=50, max. length=50, 9844728 kept reads (2813 discarded)
right reads: min. length=50, max. length=50, 10021563 kept reads (3496 discarded)
[2014-01-06 17:31:34] Building transcriptome data files..
[2014-01-06 17:31:35] Building Bowtie index from genes.fa
[2014-01-06 17:31:52] Mapping left_kept_reads to transcriptome genes with Bowtie
[2014-01-06 17:35:48] Mapping right_kept_reads to transcriptome genes with Bowtie
[2014-01-06 17:39:51] Resuming TopHat pipeline with unmapped reads
[2014-01-06 17:39:52] Mapping left_kept_reads.m2g_um to genome genome with Bowtie
[2014-01-06 17:40:13] Mapping left_kept_reads.m2g_um_seg1 to genome genome with Bowtie (1/2)
[2014-01-06 17:40:21] Mapping left_kept_reads.m2g_um_seg2 to genome genome with Bowtie (2/2)
[2014-01-06 17:40:29] Mapping right_kept_reads.m2g_um to genome genome with Bowtie
[2014-01-06 17:40:47] Mapping right_kept_reads.m2g_um_seg1 to genome genome with Bowtie (1/2)
[2014-01-06 17:40:55] Mapping right_kept_reads.m2g_um_seg2 to genome genome with Bowtie (2/2)
[2014-01-06 17:41:03] Searching for junctions via segment mapping
Coverage-search algorithm is turned on, making this step very slow
Please try running TopHat again with the option (--no-coverage-search) if this step takes too much time or memory.
[2014-01-06 17:43:36] Retrieving sequences for splices
[2014-01-06 17:43:37] Indexing splices
[2014-01-06 17:43:41] Mapping left_kept_reads.m2g_um_seg1 to genome segment_juncs with Bowtie (1/2)
[2014-01-06 17:43:46] Mapping left_kept_reads.m2g_um_seg2 to genome segment_juncs with Bowtie (2/2)
[2014-01-06 17:43:51] Joining segment hits
[2014-01-06 17:43:57] Mapping right_kept_reads.m2g_um_seg1 to genome segment_juncs with Bowtie (1/2)
[2014-01-06 17:44:02] Mapping right_kept_reads.m2g_um_seg2 to genome segment_juncs with Bowtie (2/2)
[2014-01-06 17:44:09] Joining segment hits
[2014-01-06 17:44:14] Reporting output tracks
[FAILED]
Error running /usr/bin/tophat_reports --min-anchor 8 --splice-mismatches 0 --min-report-intron 50 --max-report-intron 500000 --min-isoform-fraction 0.15 --output-dir tag1tag2_thout/ --max-multihits 20 --max-seg-multihits 40 --segment-length 25 --segment-mismatches 2 --min-closure-exon 100 --min-closure-intron 50 --max-closure-intron 5000 --min-coverage-intron 50 --max-coverage-intron 20000 --min-segment-intron 50 --max-segment-intron 500000 --read-mismatches 2 --read-gap-length 2 --read-edit-dist 2 --read-realign-edit-dist 3 --max-insertion-length 3 --max-deletion-length 3 --bowtie1 -z gzip -p8 --inner-dist-mean 50 --inner-dist-std-dev 20 --gtf-annotations genes.gtf --gtf-juncs tag1tag2_thout/tmp/genes.juncs --no-closure-search --no-microexon-search --sam-header tag1tag2_thout/tmp/genome_genome.bwt.samheader.sam --report-mixed-alignments --samtools=/usr/bin/samtools --bowtie2-max-penalty 6 --bowtie2-min-penalty 2 --bowtie2-penalty-for-N 1 --bowtie2-read-gap-open 5 --bowtie2-read-gap-cont 3 --bowtie2-ref-gap-open 5 --bowtie2-ref-gap-cont 3 genome.fa tag1tag2_thout/junctions.bed tag1tag2_thout/insertions.bed tag1tag2_thout/deletions.bed tag1tag2_thout/fusions.out tag1tag2_thout/tmp/accepted_hits tag1tag2_thout/tmp/left_kept_reads.m2g.bam,tag1tag2_thout/tmp/left_kept_reads.m2g_um.mapped.bam,tag1tag2_thout/tmp/left_kept_reads.m2g_um.candidates tag1tag2_thout/tmp/left_kept_reads.bam tag1tag2_thout/tmp/right_kept_reads.m2g.bam,tag1tag2_thout/tmp/right_kept_reads.m2g_um.mapped.bam,tag1tag2_thout/tmp/right_kept_reads.m2g_um.candidates tag1tag2_thout/tmp/right_kept_reads.bam
Loaded 10422 junctions

[gs512]

My patch is only about the resume function, in your case the issue seems to be different

[Parharn]

Alright! Thanks anyway!

[Parharn]

Expanding RAM solved the problem for me!

[N00bSeq]

I am getting the same error (fail when reporting output tracks) on tophat 2.0.11. I have made three mapping runs for each of 12 samples. Of these, only the ones using both --no-discordant and --no-mixed together fail (I have not tried these two parameters individually), and runs with these options fail for all 12 samples.

My reads have been trimmed, and pairs may not be of equal size. But as far as I understand, tophat should be able to handle that. At least, it certainly seems to when not using these particular parameters. Number of paired reads are around 30-40 M.

Increasing RAM did not help in my case.

[daidaobee]

I was using 2.0.8 and had the same problem.

My command line was

tophat2 -p 22 ref.hg19 R1.fastq R2.fastq

It kept failing out at the "Reporting output track". I added/took away different conditions as stated above and it still didn't work.

Then I decreased down to -p 5 and it completed without error. I have NO idea why, but this is my case.