HiSeq3000/4000 duplicate removal

[DNATECH]

Hi All,

Our Illumina FAS mentioned that in underloaded lanes on the patterned flow-cells even PCR-free libraries can generate significant numbers of duplicates (clusters swapping over the "walls" into the next empty nanowell if given enough time?).

http://dnatech.genomecenter.ucdavis.edu/2015/05/07/first-hiseq-3000-data-download/

Would you know of any software that can analyze/remove duplicates based on the coordinate information in the read headers (without any alignments)?

Thanks in advance,
Lutz

[GenoMax]

I doubt that software for this exists in the wild since you are asking for a very specific operation based on the location of the cluster for a condition that may occur with a relatively new technology (unless those sites that have had HiSeq X Ten have implemented something for this). Wonder if illumina can include mitigation for that in bcl2fastq itself.

[DNATECH]

Hi GenoMax,

Thanks! Currently there are no local duplicate metrics in the SAV viewer and no options in bcl2fastq for flagging or filtering for these cases.
People working with X Ten data get their data already aligned and might be able to use picard etc.? However the HiSeq 3000/4000 will be used for all kinds of applications were alignment based analyses are not an option.

[GenoMax]

Thanks! Currently there are no local duplicate metrics in the SAV viewer and no options in bcl2fastq for flagging or filtering for these cases.
People working with X Ten data get their data already aligned and might be able to use picard etc.? However the HiSeq 3000/4000 will be used for all kinds of applications were alignment based analyses are not an option.

There are tools to address duplicates (e.g. dedupe.sh from BBMap) but they are not cluster location/adjacency aware. Picard should be in this category too unless it has been modified.

I was wondering if this would be something illumina should implement in bcl2fastq software, specially if duplicates will be a problem with less than perfect libraries.

[SNPsaurus]

They should merge duplicate reads in contiguous wells for extra-high quality!

[pmiguel]

These would look like "optical duplicates", no?
I remember a recent thread where these were found in some MiSeq runs at high frequency.
--
Phillip

[pmiguel]

These would look like "optical duplicates", no?
I remember a recent thread where these were found in some MiSeq runs at high frequency.
--
Phillip

Ah, the thread is here:

http://seqanswers.com/forums/showthread.php?t=50115

Looks like "Picard MarkDuplicates" was used.

--
Phillip