Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • ABYSS never completes a run

    Hello Everyone,

    I am currently trying to assemble mammalian paired-end DNASeq files from Illumina using ABYSS. The problem is that ABYSS gets stuck during the Assembly and never finishes (or at least it didn't during seven days).
    The data consists of two 80GB fastq files that I fed to ABYSS 1.5.2. After days, the terminal still shows
    0: Reading `/data/reads_1.fq'...
    1: Reading `/data/reads_2.fq'...
    It always stays at this prompt if I try to read multiple files. I can see that no data is actually read into the RAM.
    When I merge those files to a single one, the program reads data to the RAM, makes it to "Finding adjacenct k-mer..." and remains there.
    I can see that the processes are still running, but no files are written to the disk. I am working on a single node with 32 Processors and 720GB RAM. The server uses SLURM and openMPI version 1.8.5.
    I have read that sometimes the eager limit of MPI is to small, so I set it to a higher value with a given formula, but that didn't solve the problem.
    The command I use:
    Code:
    sbatch --job-name=abyss --partition=hive --nodes=1 --ntasks-per-node=32 --wrap "
    abyss-pe k=51 n=10 name=test np=32 in='/data/reads_1.fq /data/reads_2.fq'
    "
    If someone had a similar problem and would like to share some thoughts, any help is aprreciated.

    EDIT
    Thanks pmiguel for the remark.
    With the "v=-vv" option, I got additional output. It seems as if ABYSS really takes that long to load the reads. In the beginning, it takes about 35 seconds to read 100.000 reads. That increases to about 1 minute and so on. I have the feeling, its the server that is so slow.
    Last edited by Haumich; 08-03-2015, 06:16 AM. Reason: Additional information

  • #2
    I suggest adding "v=-vv" (verbose mode) to your abyss-pe parameters to get a better idea of what is hanging it up. As long as you stdout and stderr are being captured to a log file, you will be able to see what the program is up to.

    Also, maybe you should update ABySS? The most recent release is 1.9.0
    --
    Phillip

    Comment


    • #3
      Originally posted by Haumich View Post
      With the "v=-vv" option, I got additional output. It seems as if ABYSS really takes that long to load the reads. In the beginning, it takes about 35 seconds to read 100.000 reads. That increases to about 1 minute and so on. I have the feeling, its the server that is so slow.
      I think, but do not know, that it is ABySS that is slow. The more reads it has to work with the slower it goes. Much worse than linear speed. Could be a hashing issue (in which case normalized reads or sub-sampled reads might help) or a memory space issue. Phillip and I are currently working on a mammalian genome and might get some insights into ABySS's performance along the way.

      BTW: The ABySS forum is now hosted on Biostars.



      In particular you could look at:

      Comment


      • #4
        Hello,

        thanks everyone for their suggestions. It seems that the problem was openMPI. At least, once I changed to mpich 3.1.3, the speed of the assembly increased greatly and the program finished in about a day.

        Comment

        Latest Articles

        Collapse

        • seqadmin
          Exploring Human Diversity Through Large-Scale Omics
          by seqadmin


          In 2003, researchers from the Human Genome Project (HGP) announced the most comprehensive genome to date1. Although the genome wasn’t fully completed until nearly 20 years later2, numerous large-scale projects, such as the International HapMap Project and 1000 Genomes Project, continued the HGP's work, capturing extensive variation and genomic diversity within humans. Recently, newer initiatives have significantly increased in scale and expanded beyond genomics, offering a more detailed...
          Today, 06:43 AM
        • seqadmin
          Best Practices for Single-Cell Sequencing Analysis
          by seqadmin



          While isolating and preparing single cells for sequencing was historically the bottleneck, recent technological advancements have shifted the challenge to data analysis. This highlights the rapidly evolving nature of single-cell sequencing. The inherent complexity of single-cell analysis has intensified with the surge in data volume and the incorporation of diverse and more complex datasets. This article explores the challenges in analysis, examines common pitfalls, offers...
          06-06-2024, 07:15 AM

        ad_right_rmr

        Collapse

        News

        Collapse

        Topics Statistics Last Post
        Started by seqadmin, 06-21-2024, 07:49 AM
        0 responses
        15 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 06-20-2024, 07:23 AM
        0 responses
        16 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 06-17-2024, 06:54 AM
        0 responses
        18 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 06-14-2024, 07:24 AM
        0 responses
        28 views
        0 likes
        Last Post seqadmin  
        Working...
        X