Unconfigured Ad

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • milesgr
    Member
    • Jun 2010
    • 34

    #1

    Qsub bwa pipeline...

    This may seem like a relatively easy question, but here goes:

    I am trying to create a relatively simple shell script that sits in a parent folder, goes through all subfolders, grabs all files with the extension .fq, and aligns them using BWA aln. The caveat is that I need to use qsub to do this or I will make the admins very unhappy

    My initial script was something along the lines of the following:

    result=(`find . -name "*.fq" -type f`)

    for i1 in ${result[@]}
    do

    qsub -q queue_name -l nodes=1pn=8 -V test.sh

    done

    It should be noted that the first portion simply gets the filenames (and paths) and the second calls test.sh, which is shown below (note that the first two variable paths are truncated for forum purposes):

    REFERENCE=/refpath
    BWA_HOME=/bwapath

    result=(`find . -name "*.fq" -type f`)

    for i1 in ${result[@]}
    do

    $BWA_HOME/bwa aln -t 8 $REFERENCE $i1 > $i1".sai"

    done

    I am simply trying to get BWA to run using the $1i filename and $i1.sai as the output name. The reason I ran the for loop here and in the initial script is because there seems to be a problem holding on to $i1. If I run the bottom script as a standalone script, BWA begins running on the first file. The key is I'm not certain how to get it to run qsub X times (where X is the number of .fq files), each time is with a different file/output. Any help would be greatly appreciated. Thanks!
  • milesgr
    Member
    • Jun 2010
    • 34

    #2
    Anyone?

    Comment

    • bjchen
      Junior Member
      • Jan 2012
      • 9

      #3
      What if you change your test.sh to just take one .fq file at a time and you pass the file name to test.sh in the for-loop of your initial script?

      Comment

      • Jon_Keats
        Senior Member
        • Mar 2010
        • 279

        #4
        If you are interested check out our pipeline (www.keatslab.org). Its a qsub based program. We build two flavours. One that reads all the fastq pairs in a folder and aligns then calls variants. The other leverages an input file that allows for proper RG tag assignment that is required for GATK variant calling. Nice thing is it will merge multiple lanes of the same library together for you.

        Comment

        • alec
          Member
          • Apr 2011
          • 18

          #5
          You'll probably find it more convenient to use a single job array than submitting many jobs in a loop.
          In your launch script do a single qsub:
          qsub -t 0-$[${#result[@]}-1] test.sh
          In test.sh the environment variable PBS_ARRAYID tells you which job is running:
          i1=${result[$PBS_ARRAYID]}

          Comment

          • milesgr
            Member
            • Jun 2010
            • 34

            #6
            Got it working using:

            sh "/filepath/test.sh $i1" | qsub -q queue_name -l nodes=1pn=8

            Not positive why I had to go about it in such a roundabout way.

            Comment

            Latest Articles

            Collapse

            • SEQadmin2
              Multiomic Liquid Biopsy: What DNA/RNA Panels Reveal That DNA Alone Can’t
              by SEQadmin2



              Tumors contain complex mixtures of cell populations that evolve in response to treatment and host-specific factors. Tissue biopsies provide only static snapshots of a tumor’s state. But what if we could follow that evolution over time using a minimally invasive technique?

              Liquid biopsies can detect tumor-derived material circulating in the bloodstream, providing a way to study treatment response, detect emerging resistance, and discover new biomarkers. Circulating
              ...
              Yesterday, 09:55 AM
            • SEQadmin2
              New Genomics Technologies Take Aim at Long-Standing Limits
              by SEQadmin2


              Researchers using sequencing and genomics tools often have to make trade-offs. They can choose between speed or scale, short reads or long-range information, or targeted panels or a view of the whole transcriptome. New technologies that have been released this year are built to address those tough choices.

              We asked six companies the same four questions to learn about their latest products. The new technologies bring a lot to the table, including rethinking sequencing
              ...
              09-28-2026, 10:25 AM

            ad_right_rmr

            Collapse

            News

            Collapse

            Topics Statistics Last Post
            Started by SEQadmin2, Yesterday, 05:39 AM
            0 responses
            14 views
            0 reactions
            Last Post SEQadmin2  
            Started by SEQadmin2, 10-13-2026, 09:56 AM
            0 responses
            15 views
            0 reactions
            Last Post SEQadmin2  
            Started by SEQadmin2, 10-09-2026, 11:14 AM
            0 responses
            27 views
            0 reactions
            Last Post SEQadmin2  
            Started by SEQadmin2, 09-29-2026, 09:51 AM
            0 responses
            78 views
            0 reactions
            Last Post SEQadmin2  
            Working...