I'm using Tophat to align ~33 million 100 bp paired end reads to the mouse genome using -p 8 and and when it gets about 1 hour into "Searching for junctions via segment mapping" memory usage jumps from a few GB to almost 20 GB which exceeds the 16GB on my computer and starts using the swap file. I'm also using the "butterfly-search" which the Tophat manual says will slow slow down the alignment but doesn't say it should use more memory. Is this amount of memory usage normal or did something go wrong?
Seqanswers Leaderboard Ad
Collapse
Announcement
Collapse
No announcement yet.
X
-
Same problem
I'm running Tophat2 on two 389MB paired end reads files, on 8 cores with 2.5G vmem per core (=20GB total) in SGE, using 16 threads.
The alignments go swimmingly (1 hour), then it takes ~19.7G of memory and 5+ hours to run the "Searching for junctions via segment mapping" step (my understanding is that the multiple threads are not used here). Then it makes it to "Mapping left_kept_reads_seg1 against segment_juncs with Bowtie2 (1/4)", but gets booted from the queue, probably due to memory usage.
If I run it with 4 cores, 16 threads, it gets booted at the "Searching for junctions via segment mapping" step (4 cores = only 10GB of memory).
I'm willing to go up to 10-12 cores, but looking for advice first. Any you care to provide would be greatly appreciated.
Thanks,
Robin
-
A slightly different memory issue
I am test driving TopHat 2.0 on a linux cluster.
I prepared 4 sample sets, which contain 13 million to 18 million 50bp paired reads. Tophat was called to process them one by one. I initialized the program with 8GB memory and 8 threads.
The first dataset was processed successfully using almost 2 hr. However, when the second dataset reached the step of "Building Bowtie index from genes.fa", the memory usage went above 18GB, so the system admin terminated my job.
So I re-started my job to only run TopHat 2.0 on dataset 2 using the same system setting. The job finished flawlessly.
I am wondering whether TopHat 2.0 has some memory issues.
Comment
-
Did anyone find a solution to this?
I'm mapping about 30 million 50bp reads to the human genome and during the "searching for junctions via segment mapping" step my memory usage sky rockets. About 2-4GB of memory is used before, then 24-25GB of memory is used and the run finally fails at the "Joining segment hits" step by exceeding my 32GB of memory. This is with the latest tophat2 and bowtie2.
Thanks for any help.
Comment
-
Any more info regarding this? I just realised that my first 14 jobs that use --coverage-search --microexon-search have gone into junction search and they have started gobbling up memory (16-20Gb per sample). If they arnt finished when the next batch of 14 or even worse third batch of 14 comes to the same position I'm not certain that the servers got enough memory...
Comment
-
Originally posted by pettervikman View PostAny more info regarding this? I just realised that my first 14 jobs that use --coverage-search --microexon-search have gone into junction search and they have started gobbling up memory (16-20Gb per sample). If they arnt finished when the next batch of 14 or even worse third batch of 14 comes to the same position I'm not certain that the servers got enough memory...
Comment
Latest Articles
Collapse
-
by seqadmin
The sequencing world is rapidly changing due to declining costs, enhanced accuracies, and the advent of newer, cutting-edge instruments. Equally important to these developments are improvements in sequencing analysis, a process that converts vast amounts of raw data into a comprehensible and meaningful form. This complex task requires expertise and the right analysis tools. In this article, we highlight the progress and innovation in sequencing analysis by reviewing several of the...-
Channel: Articles
05-06-2024, 07:48 AM -
-
by seqadmin
The field of epigenetics has traditionally concentrated more on DNA and how changes like methylation and phosphorylation of histones impact gene expression and regulation. However, our increased understanding of RNA modifications and their importance in cellular processes has led to a rise in epitranscriptomics research. “Epitranscriptomics brings together the concepts of epigenetics and gene expression,” explained Adrien Leger, PhD, Principal Research Scientist...-
Channel: Articles
04-22-2024, 07:01 AM -
ad_right_rmr
Collapse
News
Collapse
Topics | Statistics | Last Post | ||
---|---|---|---|---|
Started by seqadmin, 05-14-2024, 07:03 AM
|
0 responses
16 views
0 likes
|
Last Post
by seqadmin
05-14-2024, 07:03 AM
|
||
Started by seqadmin, 05-10-2024, 06:35 AM
|
0 responses
40 views
0 likes
|
Last Post
by seqadmin
05-10-2024, 06:35 AM
|
||
Started by seqadmin, 05-09-2024, 02:46 PM
|
0 responses
50 views
0 likes
|
Last Post
by seqadmin
05-09-2024, 02:46 PM
|
||
Started by seqadmin, 05-07-2024, 06:57 AM
|
0 responses
41 views
0 likes
|
Last Post
by seqadmin
05-07-2024, 06:57 AM
|
Comment