demultiplex FastQ files based on read names

[Latrunculia]

Hi everybody,

my name is Fabian Roger and I am a PhD student in Ecology at the university of Gothenburg.

I am at a stage where I need to submit sequences to ENA but I don't have the files in the right format. All I have are 6 FASTQ files (3 forward, 3 reverse) and a list of sequence names that assigns each read to it's corresponding sample (144 samples in total).

In order to get the files into the right format I need to do two things:

1) demultiplex the FASTQ files into separate samples (split in forward and reverse)

2) check if there are any non-biological sequences remaining.

I tried to do 1) with filterbyname.sh script from bbmap but I can't get it to work. Are there any solutions that are comparably fast that could do that?

And would you have a good suggestion how to check 2)?

Information about the reads:

We sequenced environmental samples (freshwater) with the 341F-806R bacterial 16S primers and sequenced it with 2x250 bp on Illumina MISEQ platform.

Any help would be greatly appreciated!

I attach two small files from my data.

[fanli]

You can use FastQC or similar to check for adapter sequence. What other non-biological sequence are you considering?

For the second part, a short pick-your-favorite-language script should suffice.

[Brian Bushnell]

Just a note, filterbyname.sh did not work because the names in list.txt were not exact matches; they contain trailing whitespace. It works correctly once that is removed.

Note that BBTools also contains a program called "demuxbyname" which can demultiplex reads into multiple files based on a list of prefixes or suffixes in the read names, but I'm not sure if that exactly addresses what you want to do. If the reads are barcoded by sample, it should work, in suffix mode.

[Latrunculia]

Hey again.

Thanks to both for your replies! As Brian noted (and kindly told me by mail, too) the problem was the trailing tab. It works great now!

For the non-biological sequences, I found out that there are both primers and barcodes still attached to them, so I will need to remove them.

Thanks for your help and time!

[GenoMax]

Use BBDuk.sh to remove adapters/extraneous stuff from BBMap, if you don't have a pre-determined pipeline already.