Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Help with REPET (TEdenovo, TEannot) on HPC cluster, MySQL not on all nodes

    I'm trying to use REPET v2.5 in a cluster environment using TORQUE. TEdenovo.py automatically submits jobs and I want to know how to control which nodes the jobs are submitted to. TEdenovo uses MySQL to work with the data but MySQL is only installed on some nodes in this cluster. I don't have permission to do much on the compute nodes and there is practically zero admin support here for this machine. In the TEdenovo.cfg file there is a 'resources:' parameter, but I can't find any documentation about possible values for it. Just one example: "mem_free=8G". Does anyone know the syntax for this parameter or how to control where REPET's pipelines submit jobs?

    Thanks
    Matt

  • #2
    @Matt: I don't think we can give you a good answer not knowing how your cluster is set up and what queues you have available for submitting jobs.

    Your best bet may be to find a job queue that only targets the nodes that have access to MySQL since without that it sounds like your program will not run.

    Comment


    • #3
      There are 10 compute nodes, two with a queue name of "performance" and two with a queue name of "default". I'm allowed to submit jobs to any queue. But MySQL is actually only installed correctly on the head node...and when I run step 1 of TEdenovo.py it runs on the head node, but the blaster part of step 2 goes to some compute node. Not sure if that helps. I'm pretty new to clusters. Is there any other info I can get that might help you understand?

      I wonder if I can run the pipeline on the head node. I may be able to get MySQL server running on one of the compute nodes, but I want to know if it's possible to specify which ones they run on. I don't know if I will be able to install MySQL Cluster.

      Comment


      • #4
        Ok it seems I can specify which node I want to run in using the same syntax used to specify the resource_list (the -l option) for the TORQUE qsub command in the TEdenovo.cfg resources: sections:

        resources: host=computenode-0-8

        My qsub man doesn't have all the possible resource_list arguments but I found them here: http://www.adaptivecomputing.com/wp-...orial_qsub.pdf

        Comment


        • #5
          The problem is if MySQL is only on the head node then jobs you submit to the nodes are going to fail. Your admins may not appreciate you running jobs on the headnode especially if you use a lot of resources and bring the head node to its knees. I suggest you try to talk with the local admins to see what would be the best solution.

          Comment

          Latest Articles

          Collapse

          • seqadmin
            Recent Advances in Sequencing Technologies
            by seqadmin







            Innovations in next-generation sequencing technologies and techniques are driving more precise and comprehensive exploration of complex biological systems. Current advancements include improved accessibility for long-read sequencing and significant progress in single-cell and 3D genomics. This article explores some of the most impactful developments in the field over the past year.

            Long-Read Sequencing
            Long-read sequencing has...
            12-02-2024, 01:49 PM
          • seqadmin
            Genetic Variation in Immunogenetics and Antibody Diversity
            by seqadmin



            The field of immunogenetics explores how genetic variations influence immune responses and susceptibility to disease. In a recent SEQanswers webinar, Oscar Rodriguez, Ph.D., Postdoctoral Researcher at the University of Louisville, and Ruben Martínez Barricarte, Ph.D., Assistant Professor of Medicine at Vanderbilt University, shared recent advancements in immunogenetics. This article discusses their research on genetic variation in antibody loci, antibody production processes,...
            11-06-2024, 07:24 PM

          ad_right_rmr

          Collapse

          News

          Collapse

          Topics Statistics Last Post
          Started by seqadmin, 12-02-2024, 09:29 AM
          0 responses
          139 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 12-02-2024, 09:06 AM
          0 responses
          50 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 12-02-2024, 08:03 AM
          0 responses
          38 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 11-22-2024, 07:36 AM
          0 responses
          70 views
          0 likes
          Last Post seqadmin  
          Working...
          X