merging two fastq files from exome sequencing

[Naarkhoo]

One sample, is whole exome sequenced, twice to reach the required coverage as these samples had some issues. The sequencing in both samples are same (batch, machine and the vendor) I wonder, if it makes sense to merge these two fastq files at the beginning in order to reach the adequate coverage, and do variant calling once afterward ?! if so, how should I merge them.

I appreciate other suggestion for this case, thanks

[Shingo]

I often merge fastq files from same sample but different sequence run like this using unix command.

Single-end
cat same_sample_1st.fastq same_sample_2nd.fastq > merged.fastq

Paired-end
cat same_sample_1st_R1.fastq same_sample_2nd_R1.fastq > merged_R1.fastq
cat same_sample_1st_R2.fastq same_sample_2nd_R2.fastq > merged_R2.fastq

I hope this is the answer.

[Apexy]

Intuitively, merging would be appropriate given that you will need to map reads to a reference. When you say "samples had some issues", I begin to wonder what kind of issue and how this might affect results (read quality, contamination, subtle error profile). I am not particularly use to SNP analysis but coverage around a nucleotide (mapped read density) would normally play a rule is deciding what is a variant.

"..........The sequencing in both samples are same ".... are these sample different? is yes, then you may not merge files.

cat 1.fastq 2.fastq > merge.fastq should work. if not then use the following:

#!/usr/bin/perl
use strict;
use warnings;

if (@ARGV != 3 ) {
#pirnts this message and note on usage
print STDERR "\nusage: perl fastq_concatenate.pl fastq1 fastq2 output\n\n";
exit(1);
}

my $fq1 = $ARGV[0];
my $fq2 = $ARGV[1];
my $fq3 = $ARGV[2];
open (PE, $fq1) || &ErrorMessage($fq1);
open (PF, $fq2) || &ErrorMessage($fq2);
open (PH, ">$fq3") || &ErrorMessage($fq3);

while(<PE>) {
print PH $_;
}
close PE;
while(<PF>) {
print PH $_;
}
close PF;
close PH;

sub ErrorMessage {
my $err = shift;
return "Fatal error: $err does not exist\n";
exit(1);
}


Hope that helps

[rohan_1925]

Concatenate paired end samples

Hi Shingo,

I have 50 samples,each having paired end files,Do you know of using "cat" command of merging samples in one command? for example merging the 2 files below in one command using some regular expression?

Else I have to merge each sample one by one.

Thanks,
Rohan

I often merge fastq files from same sample but different sequence run like this using unix command.

Single-end
cat same_sample_1st.fastq same_sample_2nd.fastq > merged.fastq

Paired-end
cat same_sample_1st_R1.fastq same_sample_2nd_R1.fastq > merged_R1.fastq
cat same_sample_1st_R2.fastq same_sample_2nd_R2.fastq > merged_R2.fastq

I hope this is the answer.

[Shingo]

Hi Rohan,

I usually use illumina HiSeq. Output fastq files from HiSeq are compressed by *.gz and their formats are “sample_name”_”index”_”lane_no”_”mate_no”_”file_no”.fastq.gz. I think the perl’s regular expression of the HiSeq fastq is
$file_name =~ /^(.+)_([ATGC]{6,})_(L\d\d\d)_(R1|R2)_(\d\d\d).fastq.gz$/.

If you use illumine HiSeq and the format is same as above, the attached perl script may be useful. fastqs will be concatenated by each sample and the result files will be moved to ‘original_fastq’ directory in the current.

Please try below.

00_zcat_fastq_files.pl [Top directory in which there are all FASTQ files (must be *.fastq.gz)]

It is OK that fastqs of samples divided into different sub directories.

Top directory
|
|- Sample_1_dir (1_*_R1.001.fastq, 1_*_R1.002.fastq, 1_*_R2.001.fastq, 1_*_R2.002.fastq, …)
|- Sample_2_dir (2_*_R1.001.fastq, 2_*_R1.002.fastq, 2_*_R2.001.fastq, 2_*_R2.002.fastq, …)
|- Sample_3_dir
.
.

I hope you get a solution.

Shingo

Hi Shingo,

I have 50 samples,each having paired end files,Do you know of using "cat" command of merging samples in one command? for example merging the 2 files below in one command using some regular expression?

Else I have to merge each sample one by one.

Thanks,
Rohan

[Shingo]

Hi Rohan,

Sorry, I forgot to attach the file.

Shingo

Hi Shingo,

I have 50 samples,each having paired end files,Do you know of using "cat" command of merging samples in one command? for example merging the 2 files below in one command using some regular expression?

Else I have to merge each sample one by one.

Thanks,
Rohan

[rohan_1925]

Hi Shingo,

Thanks for your quick response and the script.Following are some of my sample files.


51772BL1_R1.fastq
51772BL1_R2.fastq
51805BL1_R1.fastq
51805BL1_R2.fastq
52451BL1_R1.fastq
52451BL1_R2.fastq

I think I would have to modify the regex in that case.

Thanks,
Rohan

[Shingo]

Hi Rohan,

51772BL1_R1.fastq
51772BL1_R2.fastq
51805BL1_R1.fastq
51805BL1_R2.fastq
52451BL1_R1.fastq
52451BL1_R2.fastq

These files look like BL1 is the sample name and the forward numeric is serial number. Is that correct? If so, the regex should be $file_name =~ /^(\d+)(\w+)_(R1|R2).fastq$/.
I modified the previous script. Please try it. The new script doesn’t read *.fastq.gz just read *.fastq.

Shingo

[volks]

what is the benefit of merging the fastq files? you could merge bam files, e.g. using Picard MergeSamFiles or MarkDuplicates (if required, can be performed in a single step). then you could remove bad quality lanes completely without redoing the alignment.