Dedupe on assembled RNA-Seq?

[DrYak]

Hi,

I am trying to get rid of "redundant" sequences from a trinity assembly. I used dedupe.sh to get rid of duplicates in the illumina source files and got a very good result.

After assembling with trinity I get 85497 output sequences. If I cluster these with cd-hit at 95% I get 69413 clusters (mostly with >99% identity).

How can I extract a single sequence from each cluster (the longest I assume)? I'm not sure how to go from the cd-hit clstr file to getting the largest sequence of each cluster out of my assembled fasta file...

I tried to use dedupe on the assembled file but it only removed 2 sequences (which I assume were identical). What flag would I set to remove duplicates at the 99% identity level?

Thank you in advance.

[DrYak]

Hi,

Well, I found (to my chagrin) that cd-hit has an aux tools package containing the cd-hit-dup tool.

I do not, however, get the same results using cd-hit-est and cd-hit-dup.

If I use cd-hit with the following parameters:

cd-hit-est -i in.fasta -o out -c 0.95 -n 10 -d 0 - T 20

I get 85497 finished 69413 clusters

i.e. 69413 clusters from 85497 starting sequences.

If I use cd-hit-dup with the following parameters:

cd-hit-dup -i in.fasta -o out-nodupes.fasta -m false -e 0.05 -f true

Which as far as I know should have the same similarity cut-off (95%) and remove smaller sequences (-m false) and chimeras, I get:

Number of reads: 85497
Number of clusters found: 82927
Number of chimeric clusters found: 6

i.e 82921 clusters from 85497 starting sequences.

Can someone suggest an explanation for the such a huge difference?

Thanks in advance.

[mastal]

I think what you want is software that calls a consensus sequence from each cluster, rather than dedupe.