![]() |
GraphPad Home
Support
Quick Answers Database
| How can I get exact P values for the post tests that follow one-way or two-way ANOVA? FAQ# 189 | |
|
This sounds like a simple question, but in fact it can only be answered by discussing some fundamental, almost philosophical, questions. The distinction between P values and statistical hypothesis testing When you make one comparison, it is easy to gloss over the difference between reporting a P value and using statistical hypothesis testing to report a conclusion of whether or not that difference is "statistically significant". But the two are somewhat distinct:
Multiple comparisons applies to a set of comparisons, and not to any one particular comparison When you make several comparisons at once, there is a huge distinction between P values and statistical hypothesis testing. The methods of statistical hypothesis testing can be adjusted to account for the fact you are making multiple comparisons at one time. When you make one comparison, you usually set alpha to 5% which means: If the null hypothesis is true, there is a 5% chance of ending up with a 'statistically significant' result just by chance. When you are making multiple comparisons, you can set things up so that 5% value applies to the entire family of comparisons. The 5% is said to be the family-wise error rate. This means that if all the null hypotheses were true, there is a 5% chance that one or more of the differences will be 'statistically significant' simply due to random variation, leaving a 95% chance that all the comparisons will be 'not significant'. Reporting an individual P value, by its very nature, is a way to express the strength-of-evidence for that one comparison. It doesn't really make a lot of sense to compute individual P values (but see below) Prism does report the results at several different significance levels (0.05, 0.01, 0.001). Multiplicity adjusted P values Some have proposed adjusting the P value for multiple comparisons. A P value is calculated for each comparison, taking into account the others. The trick is knowing how to interpret these P values. Each multiplicity adjusted P value is the significance level at which that particular comparison would just barely be considered statistically significant. More on adjusted P values. t tests after one-way ANOVA, without correcting for multiple comparisons An alternative approach is to compute a P value for each comparison, without adjusting for multiple comparisons. This is sometimes called the unprotected Fisher's Least Significant Difference test. More.
|
|
| Keywords: Tukey, newman keuls, posthoc, Dunnett, exact P value, multiple comparison | |
|
* * * * * Copyright 2002 - 2009, GraphPad Software Inc. All rights reserved. Please email us comments and corrections. |
|