Please enable JavaScript to view this site.

If you don't need to make a decision from each comparison, you don't need each comparison to be reported as "statistically significant" or not. In this situation, ignore the conclusions about statistical significance and the P values. Instead focus  on how large each difference is and how wide each confidence interval is. When thinking about confidence intervals, you need to think about how large a difference you would consider to be scientifically important. How small a difference would you consider to be scientifically trivial? Use scientific judgment and common sense to answer these questions. Statistical calculations cannot help, as the answers depend on the context and goals of the experiment.

If you do want to focus on confidence intervals, then make sure you pick a multiple comparisons method that can report them: The methods of Tukey, Dunnett, Bonferroni, and Sidak.

Note that the confidence intervals reported with multiple comparisons tests (except for Fisher's LSD) adjust for multiple comparisons. They are sometimes called simultaneous confidence intervals.  If you make the usual assumptions and run lots of experiments, you expect that in 95% of these experiments all 95% confidence intervals will contain the true population values, which leaves 5% of experiments where any one or more of the intervals do not include the population value.

 

 

© 1995-2019 GraphPad Software, LLC. All rights reserved.