pattern of p-values for DE testing for RNA-Seq data

[alittleboy]

I have a very general question on roughly checking the results of DE tests for RNA-Seq data.

After a proper comparison of a control and treatment group using, say, edgeR, DESeq, etc., we can plot the histogram of raw p-values and see if there is any strange pattern there. I think if there is strong signal in the treatment group, and the test is good, then the DE test raw p-values should have a histogram that looks like this: it is used to illustrate FDR methodology, but I think maybe in general, we should always observe such a p-value histogram. Otherwise, if there are so few or so many very small p-values, then there may be something wrong with the test itself?

Thanks for sharing your thoughts and your experience :0

[sdriscoll]

I'm sorry if I misunderstand...but are you arguing that for all experiments we should see the same proportion of statistically significant DE genes?

[alittleboy]

I'm sorry if I misunderstand...but are you arguing that for all experiments we should see the same proportion of statistically significant DE genes?

Well, I guess not, but I think a histogram like what the link points to should be more reasonable... What do you think? Did you ever plot the p-value histograms for the DE test results? THANKS

[rskr]

That could be one possible outcome, if the two groups you are comparing are sufficiently similar and you have enough data, and your statistics were powerful enough.

However, it could indicate another problem that you don't have enough samples or reads in the samples to produce a significant result in many of your genes if there were in fact a difference. Also your statistics may not take into account the counts for example, and so may not be powerful enough to discern a difference. Maybe you could plot the fold change histogram, and see if that is similar to your DE p-value plot, to get some idea. If you have a large number of high fold changes that are insignificant, then you probably have a problem, with concluding that there are only supposed to be a few significantly differentially expressed genes, though there may be nothing wrong with the significance of the genes found.

[sdriscoll]

I was going to add...the assumption about the p-value histogram sounds correct if you're making a comparison between controls. If you're doing a real experiment and 10% of the measurable genes really are misexpressed then the histogram will be different and unique to that experiment.