Pros and cons of the three methods used to control the FDR 

Pros and cons of the three methods used to control the FDR 


Prism offers three methods to control the FDR that differ in power, simplicity and assumptions.
This method was developed first, and is still the standard. It assumes that "test statistics are independent or positive dependent". This seems to mean that while it is OK that some of the comparisons are positively correlated (if one is low, the others tend to be low), the method does not work well if some comparisons are negatively correlated (if one is low, others tend to be high).
We offer this method because it is the standard.
Twostage stepup method of Benjamini, Krieger and Yekutieli (2).
This method relies on the same assumption as the Benjamini and Hochberg method, but it is a more clever method. It first examines the distribution of P values to estimate the fraction of the null hypotheses that are actually true. It then uses this information to get more power when deciding when a P value is low enough to be called a discovery.
The only downside of this method is that the math is a bit more complicated, so it is harder to use if you were doing the calculations by hand.
The improved adaptive method of Benjamini, Krieger and Yekutieli has more power than the Benjamini and Hochberg method, while making the same assumptions, so we recommend it.
The paper that describes this metnod (2) describes several methods. Prism uses the method defined in section 6 , the twostage linear stepup procedure.
This method requires no assumptions about how the various comparisons correlate with each other. But the price of this is that is has less power, so identifies fewer comparisons as being a discovery. Another way of saying this is that the method is very conservative.
1.Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological) 289–300 (1995).
2.Benjamini, Y., Krieger, A. M. & Yekutieli, D. Adaptive linear stepup procedures that control the false discovery rate. Biometrika 93, 491–507 (2006).
3.Benjamini, Y., & Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of statistics, 1165–1188.