This article describes how to perform post tests following two-way ANOVA using the Bonferroni method as detailed in pages 741-744 and 771 in J Neter, W Wasserman, and MH Kutner,
Applied Linear Statistical Models
, 3rd edition, Irwin, 1990.
For each comparison, the confidence interval is:
To determine significance levels, calculate for each comparison:
The variables are the same as used in the confidence interval calculations. But notice the key difference. Here, you calculate a t ratio for each comparison, and then use it to determine the significance level as explained in the next paragraph. When computing a confidence interval, you choose a confidence level (95% is standard) and use that to determine a fixed value from the t distribution, which we call t*. Note that the numerator is the absolute value of the difference between means, so the t ratio will always be positive.
To determine the significance level, compare the values of the t ratio computed for each comparison against the standard values, which we abbreviate t * . For example to determine whether the comparison is significant at the 5% level (P<0.05), compare the t ratios computed for each comparison to the t * value calculated for a confidence interval of 95% (equivalent to a significance level of 5%, or a P value of 0.05) corrected for the number of comparisons and taking into account the number of degrees of freedom. As shown above, this value is 3.863. If a t ratio is greater than t * , then that comparison is significant at the 5% significance level. To determine whether a comparison is significant at the stricter 1% level, calculate the t ratio corresponding to a confidence interval of 99% (P value of 0.01) with six comparisons and six degrees of freedom. First divide 0.01 by 6 (number of comparisons), which is 0.001667. Then use the Excel formula =TINV(0.001667,6) to find the critical t ratio of 5.398. Each comparison that has a t ratio greater than 5.398 is significant at the 1% level.