Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Galaxy configuration with SGE cluster

    Hi all,

    We are using an instance of galaxy workflow management system (http://galaxy.psu.edu/) that will launch jobs on our local SGE based cluster. The initial setup went smooth and the Galaxy seemed to be working fine with the cluster and MySQL database.
    The issue we are facing is that sometimes in a workflow, a second step queued job starts without waiting for finishing the first step. I assume it is related to our galaxy config file and we might misconfigured it somewhere.
    Below is our galaxy config file (univers_wsgi.ini).

    If someone here has experience with this tool, any help would be greatly appreciated.

    Code:
    [server:main]
    use = egg:Paste
    host = 0.0.0.0
    use_threadpool = True
    threadpool_workers = 10
    
    [filter:gzip]
    use = egg:Paste#gzip
    
    [filter:proxy-prefix]
    use = egg:PasteDeploy#prefix
    prefix = /galaxy
    
    [app:main]
    paste.app_factory = galaxy.web.buildapp:app_factory
    database_connection = mysql://galaxy:galaxy@localhost/galaxydb1?unix_socket=/var/lib/mysql/mysql.sock
    database_engine_option_pool_recycle = 7200
    file_path = database/files
    new_file_path = database/tmp
    smtp_server = /usr/sbin/sendmail
    mailing_join_addr = 
    error_email_to = 
    brand = NGS 
    static_enabled = True
    static_cache_time = 360
    static_dir = %(here)s/static/
    static_images_dir = %(here)s/static/images
    static_favicon_dir = %(here)s/static/favicon.ico
    static_scripts_dir = %(here)s/static/scripts/
    static_style_dir = %(here)s/static/june_2007_style/blue
    debug = False
    use_interactive = False
    library_import_dir = database/upload_directory
    user_library_import_dir = database/user_library_import
    allow_library_path_paste = true
    admin_users = 
    require_login = True
    allow_user_creation = False
    allow_user_deletion = True
    enable_pages = True
    track_jobs_in_database = True
    enable_job_recovery = True
    set_metadata_externally = True
    retry_metadata_internally = False
    local_job_queue_workers = 5
    start_job_runners = drmaa
    default_cluster_job_runner = drmaa:///
    cluster_job_queue_workers = 1
    
    [galaxy:tool_runners]
    biomart = local:///
    encode_db1 = local:///
    hbvar = local:///
    microbial_import1 = local:///
    ucsc_table_direct1 = local:///
    ucsc_table_direct_archaea1 = local:///
    ucsc_table_direct_test1 = local:///
    upload1 = local:///

  • #2
    Hi mpiro,

    I recommend you post your question to (and subscribe to) the galaxy-dev mailing list: http://lists.bx.psu.edu/listinfo/galaxy-dev The Galaxy team reads and responds to email on this list, and it serves a forum for discussion about the technical aspects of Galaxy as well.

    Thanks,
    J.

    Emory University
    Galaxy Team

    Originally posted by mpiro View Post
    Hi all,

    We are using an instance of galaxy workflow management system (http://galaxy.psu.edu/) that will launch jobs on our local SGE based cluster. The initial setup went smooth and the Galaxy seemed to be working fine with the cluster and MySQL database.
    The issue we are facing is that sometimes in a workflow, a second step queued job starts without waiting for finishing the first step. I assume it is related to our galaxy config file and we might misconfigured it somewhere.
    Below is our galaxy config file (univers_wsgi.ini).

    If someone here has experience with this tool, any help would be greatly appreciated.

    Code:
    [server:main]
    use = egg:Paste
    host = 0.0.0.0
    use_threadpool = True
    threadpool_workers = 10
    
    [filter:gzip]
    use = egg:Paste#gzip
    
    [filter:proxy-prefix]
    use = egg:PasteDeploy#prefix
    prefix = /galaxy
    
    [app:main]
    paste.app_factory = galaxy.web.buildapp:app_factory
    database_connection = mysql://galaxy:galaxy@localhost/galaxydb1?unix_socket=/var/lib/mysql/mysql.sock
    database_engine_option_pool_recycle = 7200
    file_path = database/files
    new_file_path = database/tmp
    smtp_server = /usr/sbin/sendmail
    mailing_join_addr = 
    error_email_to = 
    brand = NGS 
    static_enabled = True
    static_cache_time = 360
    static_dir = %(here)s/static/
    static_images_dir = %(here)s/static/images
    static_favicon_dir = %(here)s/static/favicon.ico
    static_scripts_dir = %(here)s/static/scripts/
    static_style_dir = %(here)s/static/june_2007_style/blue
    debug = False
    use_interactive = False
    library_import_dir = database/upload_directory
    user_library_import_dir = database/user_library_import
    allow_library_path_paste = true
    admin_users = 
    require_login = True
    allow_user_creation = False
    allow_user_deletion = True
    enable_pages = True
    track_jobs_in_database = True
    enable_job_recovery = True
    set_metadata_externally = True
    retry_metadata_internally = False
    local_job_queue_workers = 5
    start_job_runners = drmaa
    default_cluster_job_runner = drmaa:///
    cluster_job_queue_workers = 1
    
    [galaxy:tool_runners]
    biomart = local:///
    encode_db1 = local:///
    hbvar = local:///
    microbial_import1 = local:///
    ucsc_table_direct1 = local:///
    ucsc_table_direct_archaea1 = local:///
    ucsc_table_direct_test1 = local:///
    upload1 = local:///

    Comment


    • #3
      Thanks J. .. appreciate your advice.

      .

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Current Approaches to Protein Sequencing
        by seqadmin


        Proteins are often described as the workhorses of the cell, and identifying their sequences is key to understanding their role in biological processes and disease. Currently, the most common technique used to determine protein sequences is mass spectrometry. While still a valuable tool, mass spectrometry faces several limitations and requires a highly experienced scientist familiar with the equipment to operate it. Additionally, other proteomic methods, like affinity assays, are constrained...
        04-04-2024, 04:25 PM
      • seqadmin
        Strategies for Sequencing Challenging Samples
        by seqadmin


        Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
        03-22-2024, 06:39 AM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, 04-11-2024, 12:08 PM
      0 responses
      27 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 04-10-2024, 10:19 PM
      0 responses
      30 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 04-10-2024, 09:21 AM
      0 responses
      26 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 04-04-2024, 09:00 AM
      0 responses
      52 views
      0 likes
      Last Post seqadmin  
      Working...
      X