How is a family of comparisons defined for the Bonferroni multiple comparison test following two-way ANOVA?

Prism uses the Bonferroni correction to do multiple comparisons following two-way ANOVA. The idea is to apply the 5% significance level to an entire family of comparisons, rather than to each comparison individually. How is this family defined? The details depend on how many columns (data sets) are analyzed. 

Two data sets

For each row, Prism compares the mean in the first dataset (column) with the mean in the other. There is one comparison per row, and the Bonferroni correction defines this to be one family of comparisons. If there are five rows in your data table, the Bonferroni correction is based on K=5 comparisons in a single family of comparisons. 

Three or more data sets

For each row, Prism makes several comparisons. You can choose to either compare the mean of each column with the mean of every other column, or to compare the mean of each column with the mean of a control column.

 Prism defines the set of all comparisons, on all rows, to be a family of comparisons for the Bonferroni calculations. 

For example, let's assume that a table has five rows and four data sets, and you chose to compare each column with every other column mean. For the first row, Prism makes six comparisons (AB, AC, AD, BC, BD, CD). It similarly makes six comparisons for the other four rows. The total number of comparisons is xx 5 =30. Prism considers all 30 comparisons to be one family, and does the Bonferroni correction for 30 comparisons..

How the calculations work

Each comparisons works by computing a t ratio which equals the difference between the two means, divided by the standard error of that difference. That standard error is computed from the pooled standard deviation and the sample sizes of the two groups whose means are being compared. The critical t ratio, to define significance, is determined from the number of degrees of freedom  (residual df from the ANOVA results) and the number of comparisons made (Bonferroni correction).

[This page was edited on 17-April-2010. Before that, this page incorrectly said that Prism considered each family to be its own family, rather than all comparisons in all rows to be one family.]