r-values

[rskr]

I have been thinking about this for a while, when making some albeit simple heuristics for detecting the difference between sequencing error and heterozygosity. I noticed that traditional p-values were very sensitive to even small differences in errors if there were enough data ie H0: p<=.05 HA: p>.05, where .05 was the sequencing error, if there were enough data at a position even p=.949 would be significant. It seems that if you had enough data even the rounding errors in the CPU's would be significant. Is this a general weakness of the p-value theories in statistics, and is there any robust statistic, maybe called an r-value, that takes into account the magnitude of the difference, or is this something that can only be handled with unit based values? It seems like there should be a way to weight the statistics such that more data that is collected that it be more difficult to produce significant results.

[sdriscoll]

Yeah what you're looking for is something that essentially ignores sample size since in these tests the concept of sample size isn't correctly applied anyways. Maybe something based on binning counts and comparing histograms with a bootstrap and simple Euclidean distance.

[rskr]

Yeah what you're looking for is something that essentially ignores sample size since in these tests the concept of sample size isn't correctly applied anyways. Maybe something based on binning counts and comparing histograms with a bootstrap and simple Euclidean distance.

No, sample size is key for smaller sized samples. I was thinking maybe picking the minimum difference in means I was comfortable with, then calculating the size of the smallest sample that that could be significant then scaling the size of the sample size downward as it approaches that threshold.