How do I make next-gen SEQ data non-redundant?

[Zigster]

So sequences which are substrings of other sequences - are they redundant?
How about reverse complements?

You can use vmatch to cluster them:

http://jermdemo.blogspot.com/2009/11/using-vmatch-to-combine-assemblies.html

If not then just load a hash with your seqs as keys and your deflines as values.

[kch]

You could maybe look into the fastx_collapser tool from this package: http://hannonlab.cshl.edu/fastx_tool...mmandline.html
But I don't think it will report any counts.

Perhaps you could try using wordcount from EMBOSS (http://emboss.sourceforge.net/apps/c...wordcount.html) and setting the wordsize to your read length? This will give you a list of all unique sequences and how many there are. Then you can write a script to convert it back to fasta. However, I'm not sure how if it can handle a large fasta file.

[drio]

Not sure if this is what you want but,

I suggest you align the data first and then dump it in a BAM file. After that you can mark the PCR duplicates. The BAM will contain all the reads from your sequencing. Then you can write your own tool using any of the multiple BAM libraries to report any stats you want.

[PRJ]

I want to create a non-redundant fasta file of the data from next generation sequencing run. I would like the output fasta file to include the a count number for the number of identical reads. Does anybody have a perl script that could do this?

Thanks for your help!