Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • batch job manager?

    I suppose many have this problem: how do you efficiently utilize available CPUs, disks and memory while simultaneously running many (possibly large memory) jobs? Is there a shell based tool that can take a list of work to do and dispatch the jobs (or parts of the jobs) as resources become available? I'm sure many have just cobbled their own solution. But there must be a real, robust, open source, batch manager somewhere. It'd be great if you could specify things like number of CPUs to use, be "nice" when others are doing jobs but be a hog when nobody else is doing anything, tag jobs to only run one instance because it might use too much memory, split temp files among disks, etc. Unix/Linux would be fine but operating system independent would be okay.

    Any ideas?

  • #2
    you are looking for resource managers and batch schedulers
    Torque and Maui are good places to start your search

    Comment


    • #3
      Another option is Condor:

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Best Practices for Single-Cell Sequencing Analysis
        by seqadmin



        While isolating and preparing single cells for sequencing was historically the bottleneck, recent technological advancements have shifted the challenge to data analysis. This highlights the rapidly evolving nature of single-cell sequencing. The inherent complexity of single-cell analysis has intensified with the surge in data volume and the incorporation of diverse and more complex datasets. This article explores the challenges in analysis, examines common pitfalls, offers...
        06-06-2024, 07:15 AM
      • seqadmin
        Latest Developments in Precision Medicine
        by seqadmin



        Technological advances have led to drastic improvements in the field of precision medicine, enabling more personalized approaches to treatment. This article explores four leading groups that are overcoming many of the challenges of genomic profiling and precision medicine through their innovative platforms and technologies.

        Somatic Genomics
        “We have such a tremendous amount of genetic diversity that exists within each of us, and not just between us as individuals,”...
        05-24-2024, 01:16 PM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, Today, 07:24 AM
      0 responses
      9 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, Yesterday, 08:58 AM
      0 responses
      11 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 06-12-2024, 02:20 PM
      0 responses
      16 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 06-07-2024, 06:58 AM
      0 responses
      183 views
      0 likes
      Last Post seqadmin  
      Working...
      X