Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • ABYSS never completes a run

    Hello Everyone,

    I am currently trying to assemble mammalian paired-end DNASeq files from Illumina using ABYSS. The problem is that ABYSS gets stuck during the Assembly and never finishes (or at least it didn't during seven days).
    The data consists of two 80GB fastq files that I fed to ABYSS 1.5.2. After days, the terminal still shows
    0: Reading `/data/reads_1.fq'...
    1: Reading `/data/reads_2.fq'...
    It always stays at this prompt if I try to read multiple files. I can see that no data is actually read into the RAM.
    When I merge those files to a single one, the program reads data to the RAM, makes it to "Finding adjacenct k-mer..." and remains there.
    I can see that the processes are still running, but no files are written to the disk. I am working on a single node with 32 Processors and 720GB RAM. The server uses SLURM and openMPI version 1.8.5.
    I have read that sometimes the eager limit of MPI is to small, so I set it to a higher value with a given formula, but that didn't solve the problem.
    The command I use:
    Code:
    sbatch --job-name=abyss --partition=hive --nodes=1 --ntasks-per-node=32 --wrap "
    abyss-pe k=51 n=10 name=test np=32 in='/data/reads_1.fq /data/reads_2.fq'
    "
    If someone had a similar problem and would like to share some thoughts, any help is aprreciated.

    EDIT
    Thanks pmiguel for the remark.
    With the "v=-vv" option, I got additional output. It seems as if ABYSS really takes that long to load the reads. In the beginning, it takes about 35 seconds to read 100.000 reads. That increases to about 1 minute and so on. I have the feeling, its the server that is so slow.
    Last edited by Haumich; 08-03-2015, 06:16 AM. Reason: Additional information

  • #2
    I suggest adding "v=-vv" (verbose mode) to your abyss-pe parameters to get a better idea of what is hanging it up. As long as you stdout and stderr are being captured to a log file, you will be able to see what the program is up to.

    Also, maybe you should update ABySS? The most recent release is 1.9.0
    --
    Phillip

    Comment


    • #3
      Originally posted by Haumich View Post
      With the "v=-vv" option, I got additional output. It seems as if ABYSS really takes that long to load the reads. In the beginning, it takes about 35 seconds to read 100.000 reads. That increases to about 1 minute and so on. I have the feeling, its the server that is so slow.
      I think, but do not know, that it is ABySS that is slow. The more reads it has to work with the slower it goes. Much worse than linear speed. Could be a hashing issue (in which case normalized reads or sub-sampled reads might help) or a memory space issue. Phillip and I are currently working on a mammalian genome and might get some insights into ABySS's performance along the way.

      BTW: The ABySS forum is now hosted on Biostars.



      In particular you could look at:

      Comment


      • #4
        Hello,

        thanks everyone for their suggestions. It seems that the problem was openMPI. At least, once I changed to mpich 3.1.3, the speed of the assembly increased greatly and the program finished in about a day.

        Comment

        Latest Articles

        Collapse

        • seqadmin
          Recent Advances in Sequencing Analysis Tools
          by seqadmin


          The sequencing world is rapidly changing due to declining costs, enhanced accuracies, and the advent of newer, cutting-edge instruments. Equally important to these developments are improvements in sequencing analysis, a process that converts vast amounts of raw data into a comprehensible and meaningful form. This complex task requires expertise and the right analysis tools. In this article, we highlight the progress and innovation in sequencing analysis by reviewing several of the...
          05-06-2024, 07:48 AM
        • seqadmin
          Essential Discoveries and Tools in Epitranscriptomics
          by seqadmin




          The field of epigenetics has traditionally concentrated more on DNA and how changes like methylation and phosphorylation of histones impact gene expression and regulation. However, our increased understanding of RNA modifications and their importance in cellular processes has led to a rise in epitranscriptomics research. “Epitranscriptomics brings together the concepts of epigenetics and gene expression,” explained Adrien Leger, PhD, Principal Research Scientist...
          04-22-2024, 07:01 AM

        ad_right_rmr

        Collapse

        News

        Collapse

        Topics Statistics Last Post
        Started by seqadmin, 05-14-2024, 07:03 AM
        0 responses
        25 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 05-10-2024, 06:35 AM
        0 responses
        45 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 05-09-2024, 02:46 PM
        0 responses
        59 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 05-07-2024, 06:57 AM
        0 responses
        46 views
        0 likes
        Last Post seqadmin  
        Working...
        X