The Satterthwaite method estimates an effective denominator degrees of freedom for each fixed effect test in a linear mixed model by asking how much uncertainty there is in the estimated variance of the effect being tested. It costs considerably less than Kenward-Roger and delivers most of the same benefit.

For an overview of the choice and how it compares with the alternatives, see denominator degrees of freedom in the mixed model.

Background

Franklin Satterthwaite's idea, published in 1941 and refined in 1946, is simple and general. A linear combination of mean squares is not exactly a chi-square variable, but it can be approximated by a scaled chi-square whose degrees of freedom are chosen so the first two moments match. You have already met the idea if you have used the unequal variance (Welch) t test, where Prism reports fractional degrees of freedom for exactly this reason.

Giesbrecht and Burns (1985) and Fai and Cornelius (1996) extended the idea to fixed effect tests in mixed models. Kuznetsova and colleagues implemented it in the R package lmerTest, which is where most R users encounter it today.

The idea

Take a single contrast of the fixed effects, written l'β. A comparison between two treatment means is one example. Prism tests it with

Here b is the estimate of β. If the variance in that denominator were known, t would follow a standard normal distribution. It is not known. C is computed from V, and V is computed from variance components estimated by REML from the same data. The denominator is an estimate, and it carries its own sampling variability.

Satterthwaite's answer is to approximate the sampling distribution of that estimated variance by a scaled chi-square with ν degrees of freedom, choosing ν so the first two moments match. Then t follows a t distribution with ν degrees of freedom, approximately. Matching the moments gives

All that remains is to work out the variance of the estimated variance.

Estimating the variance of the variance

Treat the contrast variance as a function of the variance components, and take its gradient at the REML estimate. The element of that gradient belonging to the ith variance component is

Write W for the asymptotic covariance matrix of the estimated variance components, obtained by inverting the REML information matrix. The delta method then gives

and the Satterthwaite degrees of freedom for that contrast are

Read the formula and the behavior follows. The numerator is the squared variance of the contrast. The denominator measures how sensitive that variance is to the variance components, weighted by how uncertain those components are. A contrast whose variance barely depends on a poorly estimated variance component gets many degrees of freedom. A contrast that leans heavily on one gets few.

Because g depends on l, the degrees of freedom differ from one contrast to the next. That is why different rows of a multiple comparisons table can report different degrees of freedom. This is expected, not an error.

Tests with more than one degree of freedom

The main effect of a factor with three or more levels is not a single contrast. It is a hypothesis Lβ = 0 in which L has rank q, tested with

To get one denominator degrees of freedom for the whole test, the hypothesis is first split into q one-dimensional contrasts that are uncorrelated under the fitted model, using the spectral decomposition of L C L'. Each of those gets its own Satterthwaite value by the formula above. Call them ν1 through νq. They are then combined:

The F ratio is then compared with the F distribution with q and ν degrees of freedom. This is the approach of Fai and Cornelius, and it is what lmerTest and SAS both use.

What to expect in practice

•Degrees of freedom are usually fractional, and on incomplete data usually smaller than the Legacy method reports.

•Standard errors and F ratios are unchanged from the plain Wald test. Only the reference distribution changes. This is the essential difference from Kenward-Roger, which also adjusts the standard errors.

•With complete, balanced data the method reproduces the classical repeated measures ANOVA degrees of freedom, so the results match ordinary repeated measures ANOVA.

•Different contrasts in the same multiple comparisons table can carry different degrees of freedom.

Satterthwaite or Kenward-Roger?

Because Satterthwaite leaves the standard errors alone, it can still be a little anticonservative when the number of subjects is very small. Kenward-Roger addresses that as well as the degrees of freedom, which is why it is the default. Satterthwaite is the better choice when Kenward-Roger is too slow, or when you need to match an analysis run in R with lmerTest.

When to use it

•Your model is large enough that Kenward-Roger takes an uncomfortably long time, but you still want honest small-sample inference.

•You are reconciling Prism output with an R analysis that used lmerTest or emmeans.

•Your design is unbalanced or has missing values, and anything is better than the Legacy method.

•You want fractional degrees of freedom without changing the F ratios your colleagues have already seen.

Edge cases and limits

•Variance components near a boundary. When a variance component is estimated at or very near zero, the REML information matrix is poorly determined. The quantity g' W g can come out very small, which drives the estimated degrees of freedom implausibly high. Treat a suspiciously large fractional degrees of freedom as a warning about the fit, not as good news.

•Multi-parameter tests that cannot be combined. The combination step needs E to be greater than q. When it is not, the approximation has no solution and no Satterthwaite denominator degrees of freedom exists for that test. This happens only with very little data.

•A random effect removed from the model. If a random effect has an estimated SD of zero and you chose to remove the term and fit a simpler model, Satterthwaite cannot be computed. Prism falls back to the Legacy method and says so in a floating note. Choose Restrict the variance to zero and keep the term(s) to avoid this.

•Scope. The method applies to linear mixed models with Gaussian errors fit by REML. It is not available for repeated measures ANOVA based on the general linear model, for ordinary ANOVA, or when the Geisser-Greenhouse correction is selected.

Comparing with other software

Results should agree closely with lmerTest applied to an lmer fit, with emmeans using its Satterthwaite mode, and with SAS PROC MIXED using DDFM=SATTERTHWAITE. As always, small differences can remain because REML optimizers stop at slightly different places and programs parameterize the covariance structure differently.

References

1.Satterthwaite FE. Synthesis of variance. Psychometrika 1941;6(5):309-316.

2.Satterthwaite FE. An approximate distribution of estimates of variance components. Biometrics Bulletin 1946;2(6):110-114.

3.Giesbrecht FG, Burns JC. Two-stage analysis based on a mixed model: large-sample asymptotic theory and small-sample simulation results. Biometrics 1985;41(2):477-486.

4.Fai AHT, Cornelius PL. Approximate F-tests of multiple degree of freedom hypotheses in generalized least squares analyses of unbalanced split-plot experiments. Journal of Statistical Computation and Simulation 1996;54(4):363-378.

5.Kuznetsova A, Brockhoff PB, Christensen RHB. lmerTest package: tests in linear mixed effects models. Journal of Statistical Software 2017;82(13):1-26.