Unconfigured Ad

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • TBAENVS
    Junior Member
    • Jan 2016
    • 5

    #1

    GNU parallel - cutadapt with paired end reads

    Hi all,

    I have many filles which I need to quality trim using cutadapt. The sequences are paired end and the filenames are on the following format:

    sample#_Lane#_R#_.fastq.gz , where # is number.

    e.g.
    29_L001_R1.fastq, 29_L001_R2.fastq, 29_L002_R1.fastq, 29_L002_R2.fastq,
    30_L003_R1.fastq, 30_L003_R2.fastq, 30_L004_R1.fastq, 30_L004_R2.fastq, etc.

    I will use cutadapt to trim these sequences using this command:

    cutadapt -a adaptors_to_trim -A adaptors_to_trim -q 20 --minimum-length 5 -o outputfile_R1 -p outputfile_R2 inputfile_R1 inputfile_R2

    Further I would like to use GNU parallel to pipe this to use as much of my 16 cores as possible and in order to kind of loop all files into cutadapt.

    This means that for each cutadapt run I need to input R1 and R2 of all my samples but keeping sample#_Lane# seperate.

    So I was thinking of listing all input files followed by piping that into gnu parallel and there defining the R1 and R2 for each sample# in combination with lane# followed by input into cutadapt. Something like this:

    find *_L00*_R*.fastq.gz | parallel DEFINING_TWO INPUT_FILES_FROM_SAME_SAMPLE_AND_LANE_ j +0 cutadapt -a adaptors_to_trim -A adaptors_to_trim -q 20 --minimum-length 5 -o outputfile_R1 -p outputfile_R2 inputfile_R1 inputfile_R2

    Is this possible at all?

    Hope it makes sense. I will of course explain more if needed.

    Thank you very much in advance.

    Best,
    Toke
  • Roy
    Member
    • Oct 2009
    • 17

    #2
    How about:

    find *_L00*_R1.fastq.gz | sed 's/_R1.fastq.gz$//' |parallel 'cutadapt -a adaptors_to_trim -A adaptors_to_trim -q 20 --minimum-length 5 -o {}_R1_cutadapt.fastq.gz -p {}_R2_cutadapt.fastq.gz {}_R1.fastq.gz {}_R2.fastq.gz &> {}.cutadapt'

    Comment

    • TBAENVS
      Junior Member
      • Jan 2016
      • 5

      #3
      Originally posted by Roy View Post
      How about:

      find *_L00*_R1.fastq.gz | sed 's/_R1.fastq.gz$//' |parallel 'cutadapt -a adaptors_to_trim -A adaptors_to_trim -q 20 --minimum-length 5 -o {}_R1_cutadapt.fastq.gz -p {}_R2_cutadapt.fastq.gz {}_R1.fastq.gz {}_R2.fastq.gz &> {}.cutadapt'
      Thank you Roy! That worked! I really owe you a beer :-)

      I have to look into this sed option. Is this right understood: the sed 's/_R1.fastq.gz$//' indicate that the R1.fastq.gz is the part of the defined files (defined by find) that has to be substituted with what are defined by the {} ?

      Also i add -j +0 after parallel to use all cores of my server.

      Again. Thank you very much!

      Comment

      • Roy
        Member
        • Oct 2009
        • 17

        #4
        No problem.

        The sed command gets rid of the _R1.fastq.gz from the end of the filenames produced by find before they are passed to parallel. Parallel then takes each shortened filename and uses it in place of {} in the command.

        The default for parallel is -j 100%, which is 1 process per core, and is usually the optimal solution. -j 0 is defined as "run as many jobs as possible", which may result in processes fighting for resources.

        Comment

        • TBAENVS
          Junior Member
          • Jan 2016
          • 5

          #5
          You are the man!

          Thank a lot!

          Have a nice day.

          Comment

          Latest Articles

          Collapse

          • kinpict.com
            Reply to 16S/ITS Profiling vs. Shotgun Metagenomics: What Changes in Your Workflow?
            by kinpict.com
            The part about this topic feels surprisingly grounded, and the details on the page is the bit that made me slow down. I’d keep <a href="https://kinpict.com/">ai family photo generator</a> bookmarked for later.
            Yesterday, 06:37 PM
          • SEQadmin2
            16S/ITS Profiling vs. Shotgun Metagenomics: What Changes in Your Workflow?
            by SEQadmin2





            NGS has revolutionized clinical microbiology by offering new ways to identify and surveil pathogens without prior knowledge of the target. One popular approach is 16S rRNA and ITS amplicon sequencing, which targets specific microbial markers. But studies are increasingly incorporating shotgun metagenomics, a technique that sequences all DNA from a sample without targeting specific microbial markers. This approach allows more detailed taxonomic classification,...
            10-30-2026, 06:46 AM

          ad_right_rmr

          Collapse

          News

          Collapse

          Topics Statistics Last Post
          Started by SEQadmin2, 11-12-2026, 07:59 AM
          0 responses
          29 views
          0 reactions
          Last Post SEQadmin2  
          Started by SEQadmin2, 11-05-2026, 09:25 AM
          0 responses
          36 views
          0 reactions
          Last Post SEQadmin2  
          Started by SEQadmin2, 11-03-2026, 11:54 AM
          2 responses
          60 views
          0 reactions
          Last Post kinpict.com  
          Started by SEQadmin2, 10-28-2026, 09:57 AM
          0 responses
          176 views
          0 reactions
          Last Post SEQadmin2  
          Working...