Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • MetaVelvet caught in an infinite loop?

    Hello,

    I've been trying to run MetaVelvet assembly of an environmental DNA sample (short, paired end, fastq, illumina reads) on a supercomputer cluster, and have been unable to generate output. I am using Velvet 1.2.10 and MetaVelvet -1.1.01. The program will run to completion (in a fraction of a second) on very small files (containing 20,000 sequences each) for both single and paired end reads, and produce a functional meta-velvet contigs file. Vevleth and Velvetg both run successfully on files containing 40,000 sequences each. However, when running MetaVelvetg on these results, the program will run until reaching the wall timer.

    I am using the following sequence of commands:
    • mpirun -np $PBS_NP ~/bin/velvet-master/velveth $fol 31 -fastq -shortPaired $sequence1 $sequence2
    • mpirun -np $PBS_NP ~/bin/velvet-master/velvetg $fol -read_trkg yes -scaffolding no
    • mpirun -np $PBS_NP ~/MetaVelvet-1.1.01/meta-velvetg $fol -scaffolding no


    Where $sequence1 and $sequence2 point to their respective ends of each sequence, and $fol points to the output directory. I have used a different output directory with each run. I typically run the process with 6 nodes, each with 8 2.66 GHz processors and access to some allocation of 2.62 TB shared memory. I have tried up to 24 of those nodes, and MetaVelvet has never run to completion, so it seems like it's not a matter of processing power.

    When I've killed the process, the error file contained the following:
    MPI: could not run executable (case #3)
    MPI: No details available, no log files found
    /opt/torque/4.2.9/spool/mom_priv/jobs/77068.hokieone.SC: line 34: 369997 Killed
    mpirun -np $PBS_NP /home/slvt16/bin/velvet-master/velveth $fol 31 -fastq $sequence1
    MPI: could not run executable (case #3)
    MPI: No details available, no log files found
    /opt/torque/4.2.9/spool/mom_priv/jobs/77068.hokieone.SC: line 35: 374736 Killed
    mpirun -np $PBS_NP /home/slvt16/bin/velvet-master/velvetg $fol -read_trkg yes -scaffolding no

    The output log shows that the program is hanging on the ------Scafolding------ step, even when run with the '-scaffolding no' flag.

    I would like to run MetaVelvet on paired end files of ~400,000 reads each, eventually. I'm running out of ideas for troubleshooting, so any help would be appreciated!

    EDIT: Cross-Posted to https://www.biostars.org/p/150101/
    Last edited by ekkogecko; 07-08-2015, 11:50 AM. Reason: Sourced Xpost

  • #2
    Also on Biostars: https://www.biostars.org/p/150101/

    Comment


    • #3
      Originally posted by GenoMax View Post
      I've cross posted to increase visibility, is that not allowed?

      Comment


      • #4
        You can cross-post. If an answer is posted in the other forum many a times OP does not come back to this forum to indicate that a solution has been found. Link in post #2 is there as a reference for your biostars post.

        Comment


        • #5
          Do you know what "MPI: could not run executable (case #3)" is referring to? Are you running into any other limits (storage, /tmp space)? Is "369997" a torque PID? If it is, can you ask the admins to see why that process was killed?

          Comment


          • #6
            I'm not sure what "MPI: could not run executable (case #3)" means, I can send a message to the system admins for some clarification there.
            As far as storage limits, there's 100s of free gigabytes open. I've been able to assemble much larger genomes on this system using Ray, so I expect that this smaller dataset would be able to run with MetaVelvet (unless the requirements are wildly different).
            I'm not sure if "369997" is a torque PID, could you explain how to check that?

            Comment


            • #7
              It does look like a PID. I have not seen numeric identifiers for any other exposed entity in a cluster.
              Ram

              Comment


              • #8
                @ekkogecko: Are you sure metavelvet is MPI compatible? Have you tried running it without MPI?

                Comment


                • #9
                  Plus, even if it were MPI compatible, the PBS script/command may need to be modified with an option or directive to pick an MPI-enabled node.
                  Ram

                  Comment


                  • #10
                    Originally posted by RamakrishnanRS View Post
                    Plus, even if it were MPI compatible, the PBS script/command may need to be modified with an option or directive to pick an MPI-enabled node.
                    Thanks for the recommendation. As of yet, I've been unable to determine if it is truly MPI compatible. Running without MPI still fails on files >20,000 sequences. I ran a couple of test trials without MPI overnight to no avail.

                    Comment

                    Latest Articles

                    Collapse

                    • seqadmin
                      Latest Developments in Precision Medicine
                      by seqadmin



                      Technological advances have led to drastic improvements in the field of precision medicine, enabling more personalized approaches to treatment. This article explores four leading groups that are overcoming many of the challenges of genomic profiling and precision medicine through their innovative platforms and technologies.

                      Somatic Genomics
                      “We have such a tremendous amount of genetic diversity that exists within each of us, and not just between us as individuals,”...
                      05-24-2024, 01:16 PM
                    • seqadmin
                      Recent Advances in Sequencing Analysis Tools
                      by seqadmin


                      The sequencing world is rapidly changing due to declining costs, enhanced accuracies, and the advent of newer, cutting-edge instruments. Equally important to these developments are improvements in sequencing analysis, a process that converts vast amounts of raw data into a comprehensible and meaningful form. This complex task requires expertise and the right analysis tools. In this article, we highlight the progress and innovation in sequencing analysis by reviewing several of the...
                      05-06-2024, 07:48 AM

                    ad_right_rmr

                    Collapse

                    News

                    Collapse

                    Topics Statistics Last Post
                    Started by seqadmin, 05-24-2024, 07:15 AM
                    0 responses
                    198 views
                    0 likes
                    Last Post seqadmin  
                    Started by seqadmin, 05-23-2024, 10:28 AM
                    0 responses
                    220 views
                    0 likes
                    Last Post seqadmin  
                    Started by seqadmin, 05-23-2024, 07:35 AM
                    0 responses
                    229 views
                    0 likes
                    Last Post seqadmin  
                    Started by seqadmin, 05-22-2024, 02:06 PM
                    0 responses
                    13 views
                    0 likes
                    Last Post seqadmin  
                    Working...
                    X