Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Combining fastq files from paired-end WES prior to trimming and alignment

    Hi there!

    I've tried to find answers to questions that apply to my scenario, but I'm still unsure about the kosher way to approach this. I should say, I'm very new to this side of bioinformatics.

    I have data from four samples, with fastq files per read (R1 and R2) and per lane (lane 1 and lane 2)

    For example, the fastq files from my first sample:
    SF10711_9-1-22_CAGATC_L001_R1_001.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_001.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_002.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_002.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_003.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_003.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_004.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_004.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_005.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_005.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_006.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_006.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_007.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_007.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_008.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_008.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_009.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_009.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_010.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_010.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R1_011.fastq.gz SF10711_9-1-22_CAGATC_L002_R1_011.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_001.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_001.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_002.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_002.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_003.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_003.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_004.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_004.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_005.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_005.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_006.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_006.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_007.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_007.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_008.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_008.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_009.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_009.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_010.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_010.fastq.gz
    SF10711_9-1-22_CAGATC_L001_R2_011.fastq.gz SF10711_9-1-22_CAGATC_L002_R2_011.fastq.gz
    I was told generally that I should concatenate the fastq files prior to trimming and alignment because it would be easier to interpret the trimming and alignment statistics. My question then, is how can these be combined?

    I.e. Should I combine all fastq files from a sample (e.g. all of the files above into a single file), combine all files within a sample by R1 and R2, or combine all files within a sample by L1 and L2.

    Apologies for the simple questions. Again, I have done some research prior to posting this question, but still feel unsure about how to proceed.

    Thanks in advance for any insight!
    Last edited by reliscu; 05-08-2021, 12:34 PM.

  • #2
    As of joining fastq's, you can do it using bash cat command:
    cat file1.fastq.gz file2.fastq.gz ... filen.fastq.gz > joined.fastq.gz
    Generally it's better to keep R1 and R2 reads in separate files.
    Nevertheless R1 and R2 read can be interleaved in one file, you could try
    reformat.sh in1=read1.fq in2=read2.fq out=reads.fq
    from the bbmap suite reformat
    As of this whole idea. I rather advice to trim reads for each pair separately, i.e. R1 and R2 pairs together for each library. Again, you can use bbduk from bbmap suite.
    Each run is separate reaction, so it can give different problems, some reads could have more adapter contamination, for example. Or in a given file there could be more overrepresented sequences, or other bias. So each pair of read should be trimmed together and separately from read from other libraries. You can generate quality metrics using FastQC

    Comment


    • #3
      Originally posted by floem7 View Post
      As of joining fastq's, you can do it using bash cat command:
      cat file1.fastq.gz file2.fastq.gz ... filen.fastq.gz > joined.fastq.gz
      Generally it's better to keep R1 and R2 reads in separate files.
      Nevertheless R1 and R2 read can be interleaved in one file, you could try
      reformat.sh in1=read1.fq in2=read2.fq out=reads.fq
      from the bbmap suite reformat
      As of this whole idea. I rather advice to trim reads for each pair separately, i.e. R1 and R2 pairs together for each library. Again, you can use bbduk from bbmap suite.
      Each run is separate reaction, so it can give different problems, some reads could have more adapter contamination, for example. Or in a given file there could be more overrepresented sequences, or other bias. So each pair of read should be trimmed together and separately from read from other libraries. You can generate quality metrics using FastQC
      Thank you for your response!

      To clarify, I could do something like this:

      Combine all R1 fastq files from L1 >> R1_L1.fastq
      Combine all R2 fastq files from L1 >> R2_L1.fastq
      >> Then trim these two files together

      Combine all R1 fastq files from L2 >> R1_L2.fastq
      Combine all R2 fastq files from L2 >> R2_L2.fastq
      >> Then trim these two files together

      Is this correct?

      As a followup, having done the trimming, is it reasonable then to combine all the trimmed reads into a SINGLE fastq, or should the separation of libraries and/or reads be maintained?

      Thanks again.

      Comment


      • #4
        Your commands are correct.

        As a followup, having done the trimming, is it reasonable then to combine all the trimmed reads into a SINGLE fastq, or should the separation of libraries and/or reads be maintained?
        I work with sequencing since 2015 and frankly I think leaving separate fastq files are standard. I think joining fastq's is useful (after trimming) if you have more than one sequencing run for the same library*. So later you can use this one file for alignment, as opposed to using few files, what is also possible (bbwrap from bbmap suite can do that).

        * although some advice to rather join resulting bam's

        Nevertheless, I hope anyone more experienced could join this post...

        Comment

        Latest Articles

        Collapse

        • seqadmin
          Current Approaches to Protein Sequencing
          by seqadmin


          Proteins are often described as the workhorses of the cell, and identifying their sequences is key to understanding their role in biological processes and disease. Currently, the most common technique used to determine protein sequences is mass spectrometry. While still a valuable tool, mass spectrometry faces several limitations and requires a highly experienced scientist familiar with the equipment to operate it. Additionally, other proteomic methods, like affinity assays, are constrained...
          04-04-2024, 04:25 PM
        • seqadmin
          Strategies for Sequencing Challenging Samples
          by seqadmin


          Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
          03-22-2024, 06:39 AM

        ad_right_rmr

        Collapse

        News

        Collapse

        Topics Statistics Last Post
        Started by seqadmin, 04-11-2024, 12:08 PM
        0 responses
        18 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 04-10-2024, 10:19 PM
        0 responses
        22 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 04-10-2024, 09:21 AM
        0 responses
        16 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 04-04-2024, 09:00 AM
        0 responses
        47 views
        0 likes
        Last Post seqadmin  
        Working...
        X