Diginorm Algorithm

[rkizen]

Hello,

I am having trouble understanding a point made in the Diginorm paper:
http://arxiv.org/pdf/1203.4802v2.pdf

They say that Diginorm discards some terminal kmer and low-abundance isoform information but I am wondering why this is?

According to the description of the algorithm, Diginorm estimates read coverage by using the median abundance of kmers for each read and discards the read if the median abundance is above some cutoff level. This should mean that any low abundance reads would be retained. If this is true, under what situations would it discard reads pertaining to terminal kmers and low-abundance isoforms?

I suspect I am missing something here and it would be very helpful to get some outside views to get me out of this mind trap.

Thank you!

[teeniedeenie]

Hi,

You are correct in that diginorm will retain low abundance reads where abundance is estimated as the median abundance of all k-mers in the read. If you were to rank order all the k-mers in a read by its observed abundance in the dataset, the abundance would be the median value. Thus, the read would be discarded based on median abundance of the kmer abundance distribution of the read (not necessarily the terminal kmers). The k-length and read length affects how sensitive the median estimation is (as described in the paper) to i.e., sequencing errors typically found at the end of Illumina reads.

Diginorm would discard reads pertaining to terminal kmers if its was, for example, a repetitive region in a read that was observed in high abundance in the dataset. In this case, the distribution of k-mer abundances of the entire read is likely even (due to repeats) and the abundance of the terminal k-mer abundance is more likely to be the median abundance of the read.

Hope this helps!

[titusbrown]

Let's see if I can give some intuition too...

Suppose you have an undersampled region (like the terminal end of a contig, or a low-abundance splice variant) next to a bunch of highly sampled regions. Then if you had a completely correct read that crossed both the highly sampled and the low sampled region, but contained more of the highly sampled region, the median would be high, and the read would be discarded. So it really has to do with high sampling right next to low sampling -- basically what adina said about repeats.

We know how to deal with this properly and have a prototype implementation, but it isn't really ready for use yet.

[rkizen]

Thank you Adina and Titus. That does make a lot of sense now. Can I ask if the new implementation makes use of the phred scores? And do you have an estimate of when it will be released?

[titusbrown]

No short-term plans to make use of phred scores; no short-term plans on releasing the new approaches. The end-trimming problems are fairly easily solved by using a high C, like C=20 or C=50, so it's not a blocker for anyone; and for now we're trying to focus on getting the next version out. Plus pubs.