Not considering followup multiple comparisons tests (post tests), how many P values does ANOVA compute in its main calculations?
•One-way ANOVA reports only one main P value.
•Two-way ANOVA reports three P value, one for each of the two factors and one for their interaction.
•Three-way ANOVA reports seven P values, one for each of the three factors, one for each of three two-way interaction and one for the three-way interaction.
Should one take multiple comparisons into account when interpreting the main P values from two- and three-way ANOVA? Statistical tradition has been to not do any correction, and GraphPad Prism follows this tradition (so only corrects for multiple comparisons for the followup tests that compare one treatment or cell with another.
Lakens argues (1) that a correction should be applied, to prevent too many false positives.
To correct for multiple comparisons of the main ANOVA P values in Prism, you should copy all the P values from the ANOVA results table and paste into one column of a Column table. If you did a three-way ANOVA, you would copy-paste seven P values into one new column. Then run the Analyze a stack of P values analysis to correct for multiple comparisons. You can correct for multiple comparisons using Bonferroni, Holm or by controlling the false discovery rate (FDR).
1. D. Lakens, Error Control in Exploratory ANOVA's: The How and the Why. Posted in his blog, The 20% statistician.