Why does a normality test of residuals from nonlinear regression give different results than a normality test of the raw data?
Prism offers normality tests in two places:
- As part of the Column Statistics analysis. This tests the normality of a stack of data. This analysis is intended to be used for Column data tables. If you entered data onto a Grouped or XY data table with subcolumns, these are averaged, and the calculations are performed only on the set of averages.
- As part of the Nonlienar regression analysis. This tests the normality of the residuals. A residual is the distance of a value from the best-fit curve. If you entered replicate values into subcolumns, and chose the default option in nonlinear regression to fit each value individually, then the normality test is based on each individual value.
If you run both normality tests on the same data, they ask different quesitons and so give different answers.
As an example, create a new XY data table and choose the Michaelis-Menten enzyme kinetics example. There are 10 rows of data in triplicate with two missing values, so 28 Y values in all.
The graphs below show both analyses. The bottom left shows a normality test as part of nonlinear regression (a choice in the Diagnostics tab), testing the null hypothesis that the 28 residuals from the best fit curve are sampled from a Gaussian distribution. The bottom right shows the results of a normality test chosen in the Column statistics analysis, Prism first averaged the triplicates to compute ten means (one for each row). It then tests the null hypothesis that those ten means are sampled from a Gaussian distribution.
The two analyses give different results. If it makes sense to fit a curve (as it does here), then the normality test performed as part of nonlienar regression is helpful, because nonlinear regression is based on the assumption that the residuals are Gaussian. The P value is high, so you conclude that the data are consistent with the assumption that the residuals are Gaussian. In contrast, the normality test which is part of Column statistics really is not helpful. It tests whether the means of the triplicates are Gaussian. The low P value leads you to reject the assumption that the triplcates are Gaussian. But this is really not a relevant question, so the answer is not useful.