Overview

After running Multifactor ANOVA, Prism generates two main results sheets:

1.Tabular results - Contains the ANOVA table and summary statistics

2.Multiple comparisons - Contains post-hoc test results (if you requested them)

This page guides you through interpreting each section of the output and understanding what your results mean.

Tabular Results Tab

The Tabular results sheet contains several sections. This page will walk through each of them assuming an experimental design in which plant height (response) was measured under various conditions of fertilizer and watering conditions (factors).

Section 1: Table Analyzed

This section merely indicates which data table was used as input for the analysis.

Section 2: Analysis Information

This section provides general information on the analysis including whether it was an ordinary or repeated measures ANOVA (note: repeated measures not yet available in Prism for multifactor ANOVA), as well as the specified value of alpha used for the analysis.

Section 3: Source of Variation Summary

This is one of the most important sections. It shows the effect sizes and significance for each factor and interaction. Below is an example of what this section may look like in Prism

 

Source of Variation

Lists each factor and interaction tested:

•Fertilizer - Main effect of fertilizer

•Watering - Main effect of watering

•Fertilizer:Watering - Two-way interaction between fertilizer and watering

 

% of Total Variation

Provides the percentage of total variation in your response variable explained by each effect. For the example above:

•41.5% of variation in plant height is explained by fertilizer type

•36.8% explained by watering frequency

•8.1% explained by the fertilizer × watering interaction

General interpretation of these values is that large percentages (>30-40%) represent strong effects, while moderate or small percentages may represent less definite or weak effects. Importantly, because these percentages are calculated from Type III sums of squares, they may not add up to 100% (main effects can share variance in this approach). Additionally, while small percentages may represent weak effects, these effects may still be found to be statistically significant with large sample sizes.

 

η² (Eta-Squared)

Eta squared is an effect size representing the proportion of total variance explained by each effect. Mathematically, it is equivalent to:

Additionally, it's related to the percent of total variation (above) by the relationship:

General interpretation guidelines:

•η² = 0.01 - Small effect

•η² = 0.06 - Medium effect

•η² = 0.14 - Large effect

When reporting results, always consider reporting both P values and effect sizes.

Partial η² (Partial Eta-Squared)

Similar to eta squared, partial eta squared is an effect size estimate that represents the proportion of variance explained by each effect after accounting for other effects in the model. Mathematically:

General interpretation guidelines are similar to those above for eta squared, though the meaning of the interpretation is slightly different. For eta squared, an interpretation might be "Fertilizer type accounts for 41% of variance in plant height for my experiment." In contrast, a similar (but distinct) interpretation for partial eta squared would be "After accounting for the variance in plant height due to the watering methods and the interaction of watering and fertilizer type, fertilizer type alone accounts for 75% of the variance in plant height."

Cohen's f

This is a common effect size for ANOVA that represents the ratio of the between-group variability (how spread out the group means are) and the within-group variability (how spread out observations are within each group). If an effect exists, you might expect that the variability between groups would greater than the variability within groups, leading to a larger value for Cohen's f. And indeed, this is generally the interpretation of this effect size (large values of Cohen's f represent larger or stronger effects). Mathematically:

General interpretation guidelines:

•f = 0.10 - Small effect

•f = 0.25 - Medium effect

•f = 0.40 - Large effect

 

P value

The statistic that most researchers are familiar with (though not always the most well-understood value!) is the P value. This value represents the probability of observing an effect for the given factor as large (or larger) than what was determined if there was truly no effect in the population. In other words, this is the probability that these results (or even more extreme results) would have been obtained purely due to random sampling from a population in which no effect was present. This probability is then compared to the specified alpha (type I error rate) to determine whether or not an effect can be considered as "statistically significant" (though this phrasing is slowly falling out of favor with many). Alpha is typically set to a value of 0.05.

Don't make the mistake of thinking that an effect with a smaller P value (for example <0.0001) is stronger than an effect with a P value of 0.03. That's not how interpretation of P values work here. Instead, this simply suggests that you have more statistical evidence to reject the null hypothesis no effect exists. The effect size tells you how strong or weak an effect is, while any size effect can (theoretically) attain statistical significance with a large enough sample size!

P value summary

This is the familiar symbolic representation of the calculated P values using asterisks. These representations make it quick and easy to see the range in which each given P value was calculated to be without looking at the numeric P value itself. The asterisk assignments are as follows:

Section 4: The ANOVA Table

This is the traditional results view for ANOVA, and provides detailed statistical information on the analysis. While many researchers may focus on the table above with effect sizes and P values, the ANOVA table provides all of the necessary values used to calculate that information. Continuing with the plant growth example (a two-factor ANOVA with a two-way interaction), the ANOVA table may look like this:

Let's look at each section

SS (Type III) - Sum of Squares

What it is: a measure of the total variation attributed to each source

How it's calculated: for each factor or interaction, a model is fit with all components except that factor or interaction and the residual sum of squares is calculated. Then a model is fit with all components including that factor or interaction and the residual sum of squares is calculated. The reduction in the residual sum of squares when the factor or interaction is included is reported as the Type III sum of squares for that factor or interaction.

Type III Sums of Squares:

•Accounts for all other effects in the model

•Each effect is evaluated as if it's the last one entered

•Appropriate for unbalanced designs

•Standard in most statistical software (including Prism)

Note: SS values don't add up to exactly Total SS due to rounding and the nature of Type III SS.

 

DF - Degrees of Freedom

What it is: The number of independent pieces of information available to estimate each effect.

How it's calculated:

•Factor DF = (number of levels - 1)

•Interaction DF = (DF of factor 1) × (DF of factor 2)

•Residual DF = Total DF - (sum of all effect DFs)

•Total DF = (total number of observations - 1)

Why DF matters:

•Determines the shape of the F-distribution

•More DF = more precision = more statistical power

•Used in calculating F-ratios and P-values

 

MS - Mean Square

What it is: The average variation (SS) per degree of freedom for a given effect.

How it's calculated: SS / DF.

Interpretation:

•Larger MS = larger effect (per degree of freedom)

•Residual MS is the "baseline" variance (within-group variation)

•If there's no effect, MS for that factor should be similar to Residual MS

•If there's an effect, MS for that factor should be much larger than Residual MS

•The ratio of MS for a factor or interaction and the Residual MS is the F-ratio

 

F (DFn, DFd) - F-Ratio

What it is: The test statistic comparing the variance from each effect to the residual (error) variance.

How it's calculated: F = MS(effect) / MS(residual)

Interpretation:

•F = 1 means effect variance equals error variance (no effect)

•F > 1 means effect variance exceeds error variance (possible effect)

•F >> 1 means effect variance greatly exceeds error variance (strong effect)

 

P value

What it is: The probability of obtaining an F-ratio this large (or larger) if there were truly no effect (e.g. assuming the null hypothesis is true).

How it's determined:

•Based on the F-ratio and degrees of freedom

•Larger F-ratio → smaller P-value

•More DF → more precision in P-value estimation

 

Residuals

What it shows: The unexplained variation in your data (within-group variation, random error).

Interpretation:

•This represents the "noise" in your data

•Variation not explained by the effects or interactions in your model

•Ideally, residual variation should be small relative to explained variation (this results in large F statistics, i.e. large effects)

•Residual MS is used as the denominator in all F-ratios

 

Total

What it shows: The total variation in your data across all observations.

Interpretation:

•This is the variation in the data across all potential groups, factors, and interactions

•This is essentially your benchmark (without considering effects or groups, how much variation is there in the data)

•Your factors and their interactions partition this total variation

•What's not explained by factors becomes residual

Multiple Comparisons Tab

After performing an ANOVA and determining that one or more effect or interaction is statistically significant, you may want to investigate which specific groups within that effect differ from each other. This is what multiple comparisons are designed to provide. The examples discussed will rely on Tukey's method of multiple comparisons correction, but the interpretation of the results is largely the same for other methods as well.

The top section of this results sheet lists the number of families (groups of comparisons used for correcting P values), the number of comparisons per family, and the specified alpha (type I error rate) used for this set of tests.

Multiple Comparisons Results Summary

This section contains the key results for the specified multiple comparisons. The table below is an example of what this section may look like. This example table shows the results for the main effects multiple comparison of the "fertilizer" factor containing three levels ("Control", "Organic", and "Chemical").

For each comparison, this table includes:

•The names of the levels being compared

•The differences in the means of the two levels being compared

•The 95% confidence interval of the mean difference

•A summary statement indicating if the adjusted P value is smaller than the specified alpha

•(Optional) the adjusted P value (not shown in the table above)

•The symbolic summary of the P value using asterisks

For a bit more insight on how these values were calculated, the test details table provides some additional information for each comparison.

Test Details Table

This table provides the complete statistical information calculated for each comparison. For our example, the table might look something like the following.

 

Most of these results are intuitive. The Mean 1 and Mean 2 columns indicate the means of each group in the indicated comparison, while the Mean diff., SE of diff, N1 and N2 (sample sizes of each group) and the degrees of freedom provide the necessary information to calculate the test statistic and determine the P value for the comparison.