Pre-alignment repeat masking

[WhatsOEver]

Hi @all,

I'm working currently on human whole genome NGS data and have some questions about repeat masking:

1) Would you say it's advisable to do masking before mapping to the genome? Why - Why not?

2) What tools could you suggest to run masking on reads? I am familiar with - and tried already - repeatmasker/repeatmodeler but it's working damn slow on reads. I thought of doing a mapping against the repbase?! Alternatively, I could use a repeat masked genome (where repeats are removed, not only in lower case, of course) and hope that repeats are not mapped anymore.

3) Through different filtering criteria, I have to discard already ~8% of my reads before doing any alignments against the reference. I think including a repeat-filtering will increase this drastically. Is there any possibility that this will have an effect on downstream analysis? Do you include such things in your calculation when estimating required read numbers to get a certain coverage?

4) (Not directly related to the question) I have recently tried a mapping against the "new" hg38 and saw that I just got more multi-reads. Having a brief look on the data, these were mainly repetitive sequences... Has anyone observed similar things?

Thanks for reading and any suggestions

[Cytosine]

Hi,

1) I guess it depends on what you want to achieve. For resequencing runs I would use repeat masking on the genome, for CNV analyses I would not mask the genome.

2) Why mask reads? That doesn't really make sense... Mask the genome you are trying to map to. Hard-masking will prevent the reads from mapping to those regions.

3) Masking repeat regions will probably lead to more unmapped reads, but that should not affect any downstream analyses with great effect. I don't remember ever doing read number correction because of repetitive regions, I guess you could do it.

4) Did not have a look at the hg38 yet.

[dpryan]

1) Are we talking about masking the genome or the reads? You can just download the repeatmasked genome, though at least if it's only soft-masked most aligners will just ignore that. In general, I'd say it's not advisable to ignore repeat regions without good reason (it's a pretty big chunk of the genome, afterall).

2) Don't mask the reads, there's no point to that. If you want to filter out reads with long homopolymer runs or that are pure di/tri-nucleotide repeats then that'll save a little time with mapping, but you'd just be spending it with filtering.

3) Certainly if you filter out reads that would otherwise align then you're biasing against some reasons. Aligning doesn't take that long, so just attempt to align them and then filter out reads with crappy MAPQs.

4) Depending on what you included in the reference that might not be very surprising. I know the new release contains a LOT of haplotypes and also centromeric sequence, all of which will increase the mapping rate and the numbers of multimappers. That doesn't surprise me really.

[Brian Bushnell]

As Devon said, don't map to a masked genome unless you have a specific reason to do so. Often genomes are masked to increase mapping speed, as things that map to repeats are often uninformative, yet can take N times longer to map (where N is the number of repeats) compared to unique sequences; therefore, masking repeats can in some cases make things orders-of-magnitude faster without losing information. But that's mainly an issue with very large conglomerations of assorted sequences. When I was working with human NGS data, neither I nor anyone that I knew used a masked human reference for anything. That would make a few of the reads that should have mapped to repeat areas instead map to unique areas, leading to false variation calls.

[swbarnes2]

Don't mask repeated regions before mapping. If a read is repetitive, you want it to map to where it belongs. You don't want it forced to map to the wrong place, because you masked away the right place.

If it really bothers you, filter away repetitive regions from your .bam after mapping.

[WhatsOEver]

Thanks for the answers!
I'm convinced it was a bad idea
I'll do repeat-filtering after mapping then.