GraphPad Statistics Guide

Pros and cons of the three methods used to control the FDR

Pros and cons of the three methods used to control the FDR

Previous topic Next topic No expanding text in this topic  

Pros and cons of the three methods used to control the FDR

Previous topic Next topic JavaScript is required for expanding text JavaScript is required for the print function Mail us feedback on this topic!  

Prism offers three methods to control the FDR that differ in power, simplicity and assumptions.

Original method of Benjamini and Hochberg (1).

This method was developed first, and is still the standard. It assumes that "test statistics are independent or positive dependent". This seems to mean that while it is OK that some of the comparisons are positively correlated (if one is low, the others tend to be low), the method does not work well if some comparisons are  negatively correlated (if one is low, others tend to be high).

We offer this method because it is the standard.

Two-stage step-up  method of Benjamini, Krieger and Yekutieli (2).

This method relies on the same assumption as the Benjamini and Hochberg method, but it is a more clever method. It first examines the distribution of P values to estimate the fraction of the null hypotheses that are actually true. It then uses this information to get more power when deciding when a P value is low enough to be called a discovery.

The only downside of this method is that the math is a bit more complicated, so it is harder to use if you were doing the calculations by hand.  

The improved adaptive method of Benjamini, Krieger and Yekutieli  has more power than the Benjamini and Hochberg method, while making the same assumptions, so we recommend it.

The paper that describes this metnod (2) describes several methods. Prism uses the method defined in section 6 , the two-stage linear step-up procedure.

Corrected method of  Benjamini & Yakutieli (3)

This method requires no assumptions about how the various comparisons correlate with each other. But the price of this is that is has less power, so identifies fewer comparisons as being a discovery. Another way of saying this is that the method is very conservative.  

References

 

1.Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological) 289–300 (1995).

2.Benjamini, Y., Krieger, A. M. & Yekutieli, D. Adaptive linear step-up procedures that control the false discovery rate. Biometrika 93, 491–507 (2006).

3.Benjamini, Y., & Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of statistics, 1165–1188.