With the release of Prism 9.0, Prism no longer allows certain types of multiple comparisons as part of two-way ANOVA when the original data do not have replicates. Specifically, when there are no replicates in the original data, simple effects comparisons within rows and simple effects comparisons within columns are no longer available. In Prism, these "simple effects" multiple comparisons go by a number of different names, and the complete list of multiple comparisons that are no longer available when there are no replicates in the data are given here:

• "Compare each cell mean with the other cell mean in that column" - simple effects within column when there are only two rows
• "Within each column, compare rows (simple effects within columns)" - simple effects within column when there are more than two rows
• "Compare each cell mean with the other cell mean in that row" - simple effects within row when there are only two columns
• "Within each row, compare columns (simple effects within rows)" - simple effects within row when there are more than two columns

The following document describes why these comparisons are no longer available without replicates in the data, and what Prism does when opening a file created with an older version that used one of these comparisons.

Why simple effects comparisons aren't allowed without replicate data

Simple effects multiple comparisons look at data within a single row (simple effects within row) or a single column (simple effects within column). To make the following explanations a little easier to understand, let's look at the following two sets of data:

One with replicate data

One without replicate data

Let's consider what happens with each of these data sets when we want to perform a simple effects comparison within rows. What this test compares is the mean (or average) of the data in one column on a single row with the mean of the data in every other column for that same row. It's important to note that this doesn't compare means of data found in multiple different rows (the multiple comparisons performed for Row 1 only look at the mean values for data in columns in that row).  These comparisons are illustrated graphically below for each of our data sets:

Comparisons with replicate data

Comparisons without replicate data

For each data set, what the test is trying to do is to compare the mean of the values in one colored area with the mean of the values in the areas with the same color. However, to do this correctly, we also need to have some measure of uncertainty ("noise" or "error" in the data). The problem is that in the case of data without replicates, there is no way for Prism to correctly determine an estimate for this uncertainty. Without getting too technical in the math involved, this makes it invalid to consider comparing the "means" of each of these colored areas (since without replicates, you're not able to consider the uncertainty in these "means").

When there ARE replicates in the data, there's no problem: Prism will calculate the mean for each of the same colored areas, and use the replicate data to calculate a measure of uncertainty. Using this information, Prism can then perform the multiple comparison tests as expected. But…

What if I performed this sort of test in an older version of Prism when it was available?

The first thing you should realize is that the results of this test in earlier versions of Prism weren't entirely wrong, but the assumptions that were being made and the presentation of the results were misleading at best in versions prior to 9.0. To understand what this means, let's keep looking at the example above without replicates, using the simple effects within row multiple comparisons test. The results of this test prior to Prism 9.0 may have looked something like this:

Results of simple effects within row on unreplicated data (see data above)

At first glance, it may not be immediately obvious what is wrong with these results. It APPEARS as though Prism correctly analyzed the data as requested. However, a closer inspection of the data shows that - regardless of which row is analyzed - the results for each set of comparisons is the same (i.e. "Column 1 vs. Column 2" is the same for "Row 1" and "Row 2" and "Row 3" etc. The same is true for every other column comparison). This should be somewhat surprising since the data in each row was unique: the "mean difference" between the data in Column 1 and Column 2 shouldn't be the same for Row 1 and Row 2, but it is! Again, without getting too far into the math involved, the reason these results are presented this way is because Prism is using something called the "Predicted Least Squares (LS) Means" when data are unreplicated to perform these multiple comparison calculations. When calculating these values and the resulting Predicted (LS) mean difference, Prism actually uses data in multiple rows (although this may not have been obvious the way these results were presented in earlier versions). So...

How are these results reported in Prism 9.0 and newer versions?

As stated above, in Prism 9.0 and newer versions, simple effects multiple comparisons are no longer provided as an option when there are no replicates in the data. If you open results from one of these tests (without replicated data) from an older version of Prism, the results that will be shown will depend on the original test (as follows):

• "Compare each cell mean with the other cell mean in that column" performed on unreplicated data will result in no multiple comparisons being reported
• "Within each column, compare rows (simple effects within columns)" performed on unreplicated data will be replaced with the "Compare row means (main row effect)" multiple comparisons test
• "Compare each cell mean with the other cell mean in that row" performed on unreplicated data will result in no multiple comparisons being reported
• "Within each row, compare columns (simple effects within rows)" performed on unreplicated data will be replaced with the "Compare column means (main column effect)" multiple comparisons test

Note that for two of these situations, Prism does not automatically provide results from a main effects multiple comparison test. This again is due to the fact that there is no replication in the data. The main effects comparison "Compare row means (main row effect)" is only available when there are more than two rows in the original data. Similarly, the main effects comparison "Compare column means (main column effect)" is only available when there are more than two columns in the original data. When there are only two rows (or only two columns), Prism isn't able to perform the "Compare row means (main row effect)" (or "Compare column means (main column effect)") multiple comparison test, and so has no valid test to revert to when there are no replicates in the data.

Let's look at this in more detail. Consider again the example above where we had unreplicated data, and performed a simple effects within row multiple comparisons test. If we were to open a file with this specific test (on this specific data) in Prism 9.0 or later, we would be presented with the results for a main column effects multiple comparison test. A sample of the results of this test are shown below:

Main column effects multiple comparison on unreplicated data

If you compare the results for "Column 1 vs. Column 2" (or any other set of comparisons) in this test vs. the results presented in the "Simple Effects within Row on unreplicated data" above, you'll see something rather interesting: the results for any given column comparison are the same! 

The bottom line

When running a two-way ANOVA without replicated data, simple effects multiple comparisons just don't really make a whole lot of sense. As such, we removed them from the available options in Prism 9.0 and later. When opening a file with an analysis from an older version of Prism that used one of these simple effects multiple comparisons tests with unreplicated data, Prism 9.0 will instead provide the results for an appropriate main effects multiple comparisons test, and present a floating note with a simple explanation along with a link that will send you.. well.. here!