picard markduplicates on huge files

[rcorbett]

Hey folks,
I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

thanks!

[malachig]

Is the BAM file sorted already? If not, is it possible that it is during sorting that picard uses excessive memory? If that's the case, you could, presort the BAM file using SAM tools (which allows max memory usage to be specified) and then use picard to mark duplicates using the 'ASSUME_SORTED' option...

[drio]

Hey folks,
I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

thanks!

Can you show us how your data looks like and the commands you are using?