Using the post test calculator

Two-way ANOVA, also called two factor ANOVA, determines how a response is affected by two factors. For example, you might measure a response to a drug after treated with vehicle, agonist or agonist+antagonist, in both men and women.

The two-way ANOVA results test whether the column variable (gender, in this example) affects the results, whether the row variable (drug in this example) affects the results, and whether there is in interaction between the two (in this example, interaction tests whether the drug effects are different in men and women).

Multiple comparison post tests let you examine the data in more detail. Prism 6 offers many multiple comparisons tests after two-way ANOVA, so if you use Prism 6 there is no need to also use the calculator.

Prism 5 performs multiple comparisons tests following two-way ANOVA for the experimental designs biologists use most often. When there are two columns (as in this example), Prism 5 will compare the two columns at each row. For this example, Prism's built-in post tests will ask:

  • Do the control responses differ between men and women?
  • Do the agonist-stimulated responses differ between men and women?
  • Do the response in the presence of both agonist and antagonist differ between men and women?

If these questions match your experimental aims, Prism's built-in post tests will suffice. Many biological experiments compare two responses at several time points or doses, and Prism built-in post tests are just what you need for these experiments. But if you have more than two columns, Prism won't perform any post tests. And even with two columns, you may wish to perform different post tests. In this example, based on the experimental design, we want to ask the following questions: 

  1. For men, is the agonist-stimulated response different than control? (Did the agonist work?)
  2. For women, is the agonist-stimulated response different than control?
  3. For men, is the agonist response different than the response in the presence of agonist plus antagonist? (Did the antagonist work?)
  4. For women, is the agonist response different than the response in the presence of agonist plus antagonist?
  5. For men, is the response in the presence of agonist plus antagonist different than control? (Does the antagonist completely block agonist reponse?)
  6. For women, is the response in the presence of agonist plus antagonist different than control?

One could imagine making many more comparisons, but we'll make just these six. The fewer comparisons you make, the more power you'll have to find differences. You must choose the comparisons based on experimental design and the questions you care about. Ideally you should pick the comparisons before you see the data. It is not appropriate to choose the comparisons you are interested in after seeing the data. For each comparison (post test) you want to know:

  • What is the 95% confidence interval for the difference?
  • Is the difference statistically significant (P<0.05)?

To use the web calculator, you need to enter two values from the ANOVA table computed by Prism (or some other program). If you performed ordinary (not repeated measures) ANOVA (as in this example) you need to find and enter the mean square for the residuals ( 78.5 for this example) and the degrees of freedom for residuals ( 6 in this example). If you performed repeated measures two-way ANOVA, you need to enter the mean square value for 'subject' and the corresponding number of degrees of freedom. For each post test comparison, you need to enter the mean value in each group as well as the sample size for each group.

Here are the values you enter:  

Comparison Mean 1 Mean 2 N1 N2
1: Men. Agonist vs. control 176.0 98.5 2 2
2: Women. Agonist vs. control 206.5 100.0 2 2
3: Men. Agonist vs. Ag+Ant 176.0 116.0 2 2
4: Women. Agonist vs. Ag+Ant 206.5 121.0 2 2
5: Men Control vs. Ag+Ant 98.5 116.0 2 2
6: Women. Control vs. Ag+Ant 100.0 121.0 2 2

And here are the results:

 

Comparison Significant? (P < 0.05?) t
1: Men. Agonist vs. control Yes 8.747
2: Women. Agonist vs. control Yes 12.020
3: Men. Agonist vs. Ag+Ant Yes 6.772
4: Women. Agonist vs. Ag+Ant Yes 9.650
5: Men Control vs. Ag+Ant No 1.975
6: Women. Control vs. Ag+Ant No 2.370

 

Comparison Mean1 - Mean2 95% CI of difference
1: Men. Agonist vs. control + 77.5 + 43.3 to + 111.7
2: Women. Agonist vs. control + 106.5 + 72.3 to + 140.7
3: Men. Agonist vs. Ag+Ant + 60.0 + 25.8 to + 94.2
4: Women. Agonist vs. Ag+Ant + 85.5 + 51.3 to + 119.7
5: Men Control vs. Ag+Ant -17.5 -51.7 to + 16.7
6: Women Control vs. Ag+Ant -21.0 -55.2 to + 13.2

The calculations account for multiple comparisons. This means that the 95% confidence level applies to all the confidence intervals. You can be 95% sure that all the intervals include the true value. The 95% probability applies to the entire family of confidence intervals, not to each individual interval. Similarly, if the null hypothesis were true (that all groups really have the same mean, and all observed differences are due to chance) there will be a 95% chance that all comparisons will be not significant, and a 5% chance that any one or more of the comparisons will be deemed statistically significant with P< 0.05.

For the sample data, we conclude that the agonist increases the response in both men and women. Adding antagonist (plus agonist) decreases the response down to a level that is indistinguishable from the control response.

Go back to the calculator    Calculation details