Overview

If you don't change the default weighting, nonlinear regression assumes that, on average, the vertical distance of the points from the curve is the same all the way along the curve. This assumption goes by the name homoscedasticity, and Prism can test this assumption with a test for homoscedasticity.

If you have chosen to differentially weight the points, Prism assumes that the weighted distance of the points from the curve is the same all the way along the curve.  Prism tests this assumption with the test for appropriate weighting.

The null hypothesis is that you picked the right weighting scheme, so there is no correlation between the Y value of the curve and the absolute value of the weighted residual. A high P value is consistent with this hypothesis. A small P value suggests that your data violate that assumption. In this case, it might make sense to choose a more appropriate weighting scheme.

Details of the method

To run these tests, Prism follows these steps.

1.For each point, compute the  difference between the actual Y value and the Y value of the fit curve at that value of X. This is called a residual.

2.If you chose a weighting scheme, apply this scheme to each residual. If you chose relative weighting, divide each residual by the predicted value of Y. Note a confusing point. In the weighting tab, you choose how to weight the square of the residuals. So the relative weighting is shown in the dialog as dividing by Y2. Here we are weighting the residual itself, not the square of the residual, so divide by Y.

3.Compute the absolute values of all the weighted residuals.

4.Create a new table (not shown) where the X values are the absolute values of the weighted residuals and the Y values are the predicted Y values of the curve. There will be one row in this table for each point entered into the regression.

5.Compute the Spearman rank correlation and compute the corresponding P value.

This method has not been published as far as we know, but it is also used by SigmaPlot (where we learned about it). Note that SigmaPlot does exactly what we say above. Even though their manual says they use the observed Y value in step 4, they actually use the predicted value. We don't know of any publication that defines this test. If you search for the Spearman test for hereoscedasticity you'll find that is usually used with linear regression, and where the residuals are correlated with the X value, rather than the predicted Y value.