help: adjusted p-values in edgeR

[pbrand]

Hi everybody,

I am relatively new to RNA-Seq and DE statistics. We have chosen to use edgeR for DE analysis and I understood most functions so far or I could answer them via searching the literature. But now I have a problem in understanding the adjustment for multiple testing.

My questions:
1. Do I have to adjust? Are the not-adjusted p-values useless (In case I have some significant differences in DE when not-adjusted but none after adjustment)?

2. Which adjustment method is the best and why?

Thanks in advance,
philipp

[chadn737]

You should use the adjusted p-values. This controls for false positives that you get when making multiple comparisons.

EdgeR uses as its default the Benjamini-Hochberg procedure. This is also the default for other programs like DESeq. Other methods are more conservative. I would just stick with Benjamini-Hochberg.

[pbrand]

You should use the adjusted p-values.

So I can consider them as 'truly' differentially expressed?

Can I say that I do not have DE, if all not-adjusted p-values which are significant show no significance anymore after adjustment?

[chadn737]

So I can consider them as 'truly' different expressed?

Can I say that I do not have DE, if all not-adjusted p-values which are significant show no significance anymore after adjustment?

First I should ask how many replicates you have.

[pbrand]

because of our low budget, I have just two samples with pooled RNA of 40 specimen each.

[chadn737]

And you are trying to find DE between these two samples? I don't think there is much you can say about differential expression without replicates.

[pbrand]

And you are trying to find DE between these two samples? I don't think there is much you can say about differential expression without replicates.

Unfortunately, yes. But what we can do is trying to find out possible candidate DE - genes for realtime PCR. Therefore I wanted to follow the suggested way to estimate DE via estimated dispersion.

Hope that some money occurs if the results look promising to get replicates..

[chadn737]

Unfortunately, yes. But what we can do is trying to find out possible candidate DE - genes for realtime PCR. Therefore I wanted to follow the suggested way to estimate DE via estimated dispersion.

Hope that some money occurs if the results look promising to get replicates..

If you are using the data as an initial screen to get candidates that you will verify by qRT-PCR, then its fine to use the unadjusted p-values.

The qRT-PCR will either confirm these results or not. But for publication of the RNA-seq, you really do need replicates and adjusted p-values.

[pbrand]

Thanks a lot for your answers!
Philipp

[younko]

Hello chadn737

I also have similar problems.
I do not have a replicate. but I have several patients having same disease.

For example, in my case, I have 10 patients having same disease.
We treated the drug and did RNA-seq for before-drug/after-drug.

What we want to do is to find the differentially expressed genes reponsing to the drug. Instead of replications, we have 10 patients samples.. for pre/post.

I used edgeR and DESeq for this.. but I could not find the any DEG with adjusted p value .. So I am thinking to look at the p value instead of adjusted pvalue by considering the fact that we can have FP... of course...

Would it be okay?

[mikep]

Actually, what you describe are replicates. It just so happens that the biological variability is overwhelming your treatment variability.

Did you try a paired analysis?

As mentioned above, if you are going to validate candidate genes by some other method then the pvalue doesn't matter as much (though you'll waste time wading through a bunch of false positives). If you want to publish it standalone, it has to pass the correction.

[mbblack]

Hello chadn737

I also have similar problems.
I do not have a replicate. but I have several patients having same disease.

For example, in my case, I have 10 patients having same disease.
We treated the drug and did RNA-seq for before-drug/after-drug.

What we want to do is to find the differentially expressed genes reponsing to the drug. Instead of replications, we have 10 patients samples.. for pre/post.

I used edgeR and DESeq for this.. but I could not find the any DEG with adjusted p value .. So I am thinking to look at the p value instead of adjusted pvalue by considering the fact that we can have FP... of course...

Would it be okay?

As mikep said, your biological variation between patients is likely the problem, obscuring your treatment affect.

You could try simple pairwise, patient by patient, comparisons using some of the available non-parametric methods, like a Rank Product analysis (RANKPROD in R, for example uses permutation tests in the absence of replicates), the R tool GFOLD (which uses bayesion posterior prediction of fold change to calculate p-values and does not require replicates).

[younko]

Thank you for your response.(mikep and mbblack)!!

Q1 : So, are you recommending the "Wilcoxon signed-rank test " instead of GLM model from edgeR or DESeq?? This would not consider the variation like GLM anova analysis...... Do you think is it okay?

Actually, I made the glm model with EdgeR.

design <- model.matrix(~drug)

If you want to publish it standalone, it has to pass the correction.

Q2 : As I mentioned, if I don't have any DEG with correted pvalue, then, I cannot publish the result??? Or can I validate in anyway with DEG filtered with pvalue instead of adjusted p-value for publication?

[mikep]

Well I'm a fan of parametric over parametric, what I was suggesting is a better design in which you take the paired nature of the data into account

... my R is crap but something like design <- model.matrix(~drug+patientID). The paired analysis may well go a long way to removing the biological variability.

re Q2: correct, if the pvalues are not significant after correction you cannot publish alone. if you have some kind of orthagonal validation like qPCR then you can.

[mbblack]

If you take a look at the GFOLD publication, for example, and the GFOLD documentation in BioConductor, it is more sophisticated than a simple Wilcoxon test. GFOLD represents one effort to deal with the (admittedly undesirable) situtation of not have any biological replicates, yet still attempt to maintain some rigor to how differentially expressed genes are selected.

People publish differential gene expression results based on nothing but fold change, but be prepared to fully justify why you did not include replication into your study design (e.g. sometimes, clinical studies, or any study using very difficult to obtain and/or highly limited samples are simply unable to include biological replicates). You would also be best to validate at least some of the genes, at least those critical to your final interpretation or conclusions, or you'll find you are required to anyway to get past peer review.

Given that there are published methods for making the most, statistically, from no or very few replicates, you would also be best to explore them, as a reviewer familiar with them will point them out to you if you do not.