Removing duplicates from 16S RNA data

[mhadidi2002]

Hello All,

I would like to start a pipeline to analyse 16S RNA data, however I am not sure weather to include the step of removing duplicates or not. The aim of metagenomics is to compare different samples and see how the microbiota is different within samples of different conditions/time points.

After aligning, each sequence will match a sequence from a database, which will be then mapped to a bacteria and assigned taxonomy. the aim is to find how microbial content changes within each sample. If I removed duplicates, each sequence will be represented once, then taxonomy to this organism will be assigned once! so I am losing data here, which is the content of unknown bacteria in a sample with quantity for each one.

I am not sure if I am perceiving this right or not! I appreciate your comments.

Regards,
Bioinfguy

[kmcarr]

If I removed duplicates, each sequence will be represented once, then taxonomy to this organism will be assigned once! so I am losing data here, which is the content of unknown bacteria in a sample with quantity for each one.

I think you just answered your own question. No, do not remove duplicates for 16S metagenomics studies.

[mhadidi2002]

Thanks Kmcarr for the reply. I am asking because in mothur pipeline they do remove duplicates, that's why I am confused. Do you know why they are doing such so?
if you go to the link: http://www.mothur.org/wiki/MiSeq_SOP and searched by "unique.seqs" you will find it!

[kmcarr]

Thanks Kmcarr for the reply. I am asking because in mothur pipeline they do remove duplicates, that's why I am confused. Do you know why they are doing such so?
if you go to the link: http://www.mothur.org/wiki/MiSeq_SOP and searched by "unique.seqs" you will find it!

The tutorial explains why they are doing it, for efficiency. They keep a single copy of each unique sequence so that you only have to align that sequence to your reference once. But notice that in this process you also create a file with the counts of each unique sequence in the original data set which is used later in the analysis so you have not lost that abundance data. A more apt name for this is "collapsing duplicates" as opposed to "removing duplicates".

[mhadidi2002]

But notice that in this process you also create a file with the counts of each unique sequence in the original data set which is used later in the analysis so you have not lost that abundance data.

Fastx has a tool called fastq_collaper which output a fasta file like this:

>1-934
CCTACGGGAGGCAGCAGTGAGGAATATTGGTCAATGGGCGGAAGCCTGAA
>2-915
CCTACGGGAGGCAGCAGTGGGGAATATTGCACAATGGGGGAAACCCTGAT

This means that sequence no. 1 has 934 duplicates. when I use this to align against database, the aligner consider 1-934 to be the sequence ID, will not know that it has 934 duplicates. How can I use this in further analysis to keep the abundance?

[kmcarr]

Fastx has a tool called fastq_collaper which output a fasta file like this:

>1-934
CCTACGGGAGGCAGCAGTGAGGAATATTGGTCAATGGGCGGAAGCCTGAA
>2-915
CCTACGGGAGGCAGCAGTGGGGAATATTGCACAATGGGGGAAACCCTGAT

This means that sequence no. 1 has 934 duplicates. when I use this to align against database, the aligner consider 1-934 to be the sequence ID, will not know that it has 934 duplicates. How can I use this in further analysis to keep the abundance?

Are you using Mothur or are you using your own custom pipeline? If you are using your own then you will need to figure out a way to add the count information back into your analysis.

[mhadidi2002]

Are you using Mothur or are you using your own custom pipeline? If you are using your own then you will need to figure out a way to add the count information back into your analysis.

I am using my custom pipeline. so you mean that if I used mothur pipeline, they keep this in consideration to the rest of pipeline, but if I am using my pipeline I have to write a script that do this.. cool. Thnaks for the help!