Remove duplicates for differential expression with PE reads?

[ymc]

Is there a consensus now whether we should remove duplicates for PE reads?

I think for SE reads, removal is no good because of higher ratio of true duplicates to PCR duplicates.

PE reads should have a lower ratio but is it low enough to warrant duplicates removal???

Thanks in advance!

[Brian Bushnell]

You should never remove duplicates if you did not PCR-amplify. You should not amplify data that you want to measure quantitatively. Ergo, you should not remove duplicates for quantitative studies, because all duplicates are biological for unamplified data.

That said - if your data is amplified, you should just hope it all amplified about the same and still not remove duplicates. The purpose of removing duplicates is to eliminate non-independent observations of an event when doing variant-calling, as variant-callers will generally assume that the events you feed them are independent and calculate probabilities accordingly.

And yes, I would never suggest duplicate removal from SE data.

[ymc]

You should never remove duplicates if you did not PCR-amplify. You should not amplify data that you want to measure quantitatively. Ergo, you should not remove duplicates for quantitative studies, because all duplicates are biological for unamplified data.

That said - if your data is amplified, you should just hope it all amplified about the same and still not remove duplicates. The purpose of removing duplicates is to eliminate non-independent observations of an event when doing variant-calling, as variant-callers will generally assume that the events you feed them are independent and calculate probabilities accordingly.

And yes, I would never suggest duplicate removal from SE data.

Thank you very much for your detailed reply.