Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • merging two fastq files from exome sequencing

    One sample, is whole exome sequenced, twice to reach the required coverage as these samples had some issues. The sequencing in both samples are same (batch, machine and the vendor) I wonder, if it makes sense to merge these two fastq files at the beginning in order to reach the adequate coverage, and do variant calling once afterward ?! if so, how should I merge them.

    I appreciate other suggestion for this case, thanks

  • #2
    I often merge fastq files from same sample but different sequence run like this using unix command.

    Single-end
    cat same_sample_1st.fastq same_sample_2nd.fastq > merged.fastq

    Paired-end
    cat same_sample_1st_R1.fastq same_sample_2nd_R1.fastq > merged_R1.fastq
    cat same_sample_1st_R2.fastq same_sample_2nd_R2.fastq > merged_R2.fastq

    I hope this is the answer.

    Comment


    • #3
      Intuitively, merging would be appropriate given that you will need to map reads to a reference. When you say "samples had some issues", I begin to wonder what kind of issue and how this might affect results (read quality, contamination, subtle error profile). I am not particularly use to SNP analysis but coverage around a nucleotide (mapped read density) would normally play a rule is deciding what is a variant.

      "..........The sequencing in both samples are same ".... are these sample different? is yes, then you may not merge files.

      cat 1.fastq 2.fastq > merge.fastq should work. if not then use the following:

      #!/usr/bin/perl
      use strict;
      use warnings;

      if (@ARGV != 3 ) {
      #pirnts this message and note on usage
      print STDERR "\nusage: perl fastq_concatenate.pl fastq1 fastq2 output\n\n";
      exit(1);
      }

      my $fq1 = $ARGV[0];
      my $fq2 = $ARGV[1];
      my $fq3 = $ARGV[2];
      open (PE, $fq1) || &ErrorMessage($fq1);
      open (PF, $fq2) || &ErrorMessage($fq2);
      open (PH, ">$fq3") || &ErrorMessage($fq3);

      while(<PE>) {
      print PH $_;
      }
      close PE;
      while(<PF>) {
      print PH $_;
      }
      close PF;
      close PH;

      sub ErrorMessage {
      my $err = shift;
      return "Fatal error: $err does not exist\n";
      exit(1);
      }


      Hope that helps

      Comment


      • #4
        Concatenate paired end samples

        Hi Shingo,

        I have 50 samples,each having paired end files,Do you know of using "cat" command of merging samples in one command? for example merging the 2 files below in one command using some regular expression?

        Else I have to merge each sample one by one.

        Thanks,
        Rohan


        Originally posted by Shingo View Post
        I often merge fastq files from same sample but different sequence run like this using unix command.

        Single-end
        cat same_sample_1st.fastq same_sample_2nd.fastq > merged.fastq

        Paired-end
        cat same_sample_1st_R1.fastq same_sample_2nd_R1.fastq > merged_R1.fastq
        cat same_sample_1st_R2.fastq same_sample_2nd_R2.fastq > merged_R2.fastq

        I hope this is the answer.

        Comment


        • #5
          Hi Rohan,

          I usually use illumina HiSeq. Output fastq files from HiSeq are compressed by *.gz and their formats are “sample_name”_”index”_”lane_no”_”mate_no”_”file_no”.fastq.gz. I think the perl’s regular expression of the HiSeq fastq is
          $file_name =~ /^(.+)_([ATGC]{6,})_(L\d\d\d)_(R1|R2)_(\d\d\d).fastq.gz$/.

          If you use illumine HiSeq and the format is same as above, the attached perl script may be useful. fastqs will be concatenated by each sample and the result files will be moved to ‘original_fastq’ directory in the current.

          Please try below.

          00_zcat_fastq_files.pl [Top directory in which there are all FASTQ files (must be *.fastq.gz)]

          It is OK that fastqs of samples divided into different sub directories.

          Top directory
          |
          |- Sample_1_dir (1_*_R1.001.fastq, 1_*_R1.002.fastq, 1_*_R2.001.fastq, 1_*_R2.002.fastq, …)
          |- Sample_2_dir (2_*_R1.001.fastq, 2_*_R1.002.fastq, 2_*_R2.001.fastq, 2_*_R2.002.fastq, …)
          |- Sample_3_dir
          .
          .

          I hope you get a solution.

          Shingo


          Originally posted by rohan_1925 View Post
          Hi Shingo,

          I have 50 samples,each having paired end files,Do you know of using "cat" command of merging samples in one command? for example merging the 2 files below in one command using some regular expression?

          Else I have to merge each sample one by one.

          Thanks,
          Rohan

          Comment


          • #6
            Hi Rohan,

            Sorry, I forgot to attach the file.

            Shingo


            Originally posted by rohan_1925 View Post
            Hi Shingo,

            I have 50 samples,each having paired end files,Do you know of using "cat" command of merging samples in one command? for example merging the 2 files below in one command using some regular expression?

            Else I have to merge each sample one by one.

            Thanks,
            Rohan
            Attached Files

            Comment


            • #7
              Hi Shingo,

              Thanks for your quick response and the script.Following are some of my sample files.


              51772BL1_R1.fastq
              51772BL1_R2.fastq
              51805BL1_R1.fastq
              51805BL1_R2.fastq
              52451BL1_R1.fastq
              52451BL1_R2.fastq

              I think I would have to modify the regex in that case.

              Thanks,
              Rohan

              Comment


              • #8
                Hi Rohan,

                51772BL1_R1.fastq
                51772BL1_R2.fastq
                51805BL1_R1.fastq
                51805BL1_R2.fastq
                52451BL1_R1.fastq
                52451BL1_R2.fastq

                These files look like BL1 is the sample name and the forward numeric is serial number. Is that correct? If so, the regex should be $file_name =~ /^(\d+)(\w+)_(R1|R2).fastq$/.
                I modified the previous script. Please try it. The new script doesn’t read *.fastq.gz just read *.fastq.

                Shingo
                Attached Files

                Comment


                • #9
                  what is the benefit of merging the fastq files? you could merge bam files, e.g. using Picard MergeSamFiles or MarkDuplicates (if required, can be performed in a single step). then you could remove bad quality lanes completely without redoing the alignment.

                  Comment

                  Latest Articles

                  Collapse

                  • seqadmin
                    The Impact of AI in Genomic Medicine
                    by seqadmin



                    Artificial intelligence (AI) has evolved from a futuristic vision to a mainstream technology, highlighted by the introduction of tools like OpenAI's ChatGPT and Google's Gemini. In recent years, AI has become increasingly integrated into the field of genomics. This integration has enabled new scientific discoveries while simultaneously raising important ethical questions1. Interviews with two researchers at the center of this intersection provide insightful perspectives into...
                    02-26-2024, 02:07 PM
                  • seqadmin
                    Multiomics Techniques Advancing Disease Research
                    by seqadmin


                    New and advanced multiomics tools and technologies have opened new avenues of research and markedly enhanced various disciplines such as disease research and precision medicine1. The practice of merging diverse data from various ‘omes increasingly provides a more holistic understanding of biological systems. As Maddison Masaeli, Co-Founder and CEO at Deepcell, aptly noted, “You can't explain biology in its complex form with one modality.”

                    A major leap in the field has
                    ...
                    02-08-2024, 06:33 AM

                  ad_right_rmr

                  Collapse

                  News

                  Collapse

                  Topics Statistics Last Post
                  Started by seqadmin, Yesterday, 06:12 AM
                  0 responses
                  19 views
                  0 likes
                  Last Post seqadmin  
                  Started by seqadmin, 02-23-2024, 04:11 PM
                  0 responses
                  67 views
                  0 likes
                  Last Post seqadmin  
                  Started by seqadmin, 02-21-2024, 08:52 AM
                  0 responses
                  74 views
                  0 likes
                  Last Post seqadmin  
                  Started by seqadmin, 02-20-2024, 08:57 AM
                  0 responses
                  65 views
                  0 likes
                  Last Post seqadmin  
                  Working...
                  X