The Kenward-Roger method makes two corrections to a fixed effect test in a linear mixed model. It adjusts the estimated covariance matrix of the fixed effects to remove the bias caused by estimating the variance components, and it approximates the distribution of the resulting statistic by a scaled F distribution with an estimated denominator degrees of freedom. It is the most accurate of the three methods Prism offers, and the slowest.

For an overview of the choice and how it compares with the alternatives, see denominator degrees of freedom in the mixed model.

Background

Kenward and Roger published the method in 1997. Their concern was that a Wald test in a mixed model treats the estimated covariance matrix of the fixed effect estimates as though it were known exactly. It is not. It is computed from variance components that were themselves estimated from the same data.

Ignoring that has two consequences, and both push P values in the same direction. The standard errors come out too small, and the reference distribution is too generous. In large samples neither matters. With a handful of subjects the combined effect can be substantial, and tests that look convincing are not.

The method has since become the reference standard for fixed effect tests in linear mixed models. It is what SAS PROC MIXED does with DDFM=KENWARDROGER, and what the R package pbkrtest does.

What the method corrects

Prism fits the linear mixed model

where β holds the fixed effects, u holds the random effects, and θ holds the variance components. Given an REML estimate of θ, the fixed effects are estimated by generalized least squares and their covariance matrix is estimated as follows, with V evaluated at that estimate:

That estimate is too small for two separate reasons. It is evaluated at an estimate of θ rather than at the true value, and it makes no allowance at all for the fact that θ had to be estimated. Kenward and Roger correct both.

The adjusted covariance matrix

Write W for the asymptotic covariance matrix of the estimated variance components, obtained by inverting the REML information matrix. Let the subscripts i and j index the variance components, and define

The adjusted covariance matrix of the fixed effects is

Prism's mixed model uses a compound symmetry covariance structure, which is linear in the variance components. Every second derivative of V is therefore zero and the R term drops out entirely. That simplification is part of why the method is tractable here.

The adjustment is positive in the directions that matter, so standard errors get larger, F ratios get smaller, and P values get larger. This is the part of Kenward-Roger that Satterthwaite does not do.

Matching to an F distribution

For a hypothesis Lβ = 0, where L has rank q, the Wald statistic built on the adjusted covariance is

Here b is the estimate of β. Even with the corrected covariance, F* does not follow an F distribution. So Kenward and Roger choose a scale factor λ and a denominator degrees of freedom m such that λF* has approximately the right first two moments for an F distribution with q and m degrees of freedom.

Define

The method then forms two scalar summaries of how much the adjustment matters:

From those come the intermediate constants

and then the approximate mean and variance of the statistic,

Finally, define

The scale factor and the denominator degrees of freedom are then

Prism reports m as the denominator degrees of freedom and compares λF* to the F distribution with q and m degrees of freedom.

Tests with one degree of freedom

When q equals 1 the whole thing collapses to a t test. The standard error comes from the adjusted covariance matrix and the reference distribution is a t distribution with m degrees of freedom. This is the form Prism uses for multiple comparisons and for the confidence interval of each comparison, which is why different comparisons in the same table can carry different degrees of freedom.

What to expect in practice

•Denominator degrees of freedom are usually fractional, and usually smaller than the Legacy method reports.

•Standard errors are slightly larger, F ratios slightly smaller, and P values slightly larger. Kenward-Roger is more conservative than an unadjusted Wald test, which is the point of using it.

•With complete, balanced data the corrections vanish and the method reproduces the classical repeated measures ANOVA result.

•The gap between Kenward-Roger and the other methods grows as the number of subjects falls and as more values go missing.

When to use it

Kenward-Roger is the default and is worth keeping unless you have a specific reason not to. It matters most when:

•You have a small or modest number of subjects, animals, plates, or blocks.

•Your repeated measures design has missing values.

•Your design is unbalanced.

•You are working toward publication or a regulated submission where agreement with SAS or R mixed model inference matters.

•A test looks suspiciously convincing given how little data stands behind it.

You need it less when sample sizes are large, because the corrections become negligible and every method agrees.

Edge cases and limits

•Speed. The method needs derivatives of V with respect to every variance component, plus several matrix products the size of the fixed effect design. Analysis time grows quickly with the number of subjects and with the number of variance components. Three-way repeated measures models with matching on more than one factor are the slowest case. If an analysis takes too long, Satterthwaite is the natural fallback.

•Variance components near a boundary. The adjustment depends on the curvature of the REML likelihood at the estimate. When a variance component is estimated at or very near zero, that curvature is poorly determined, W is unreliable, and the adjustment can be unstable. Interpret such results with caution whichever method you choose.

•A random effect removed from the model. If a random effect has an estimated SD of zero and you chose to remove the term and fit a simpler model, Kenward-Roger cannot be computed. Prism falls back to the Legacy method and says so in a floating note. Choose Restrict the variance to zero and keep the term(s) to avoid this.

•Scope. The method applies to linear mixed models with Gaussian errors fit by REML. It is not available for repeated measures ANOVA based on the general linear model, for ordinary ANOVA, or when the Geisser-Greenhouse correction is selected.

Comparing with other software

Prism follows Kenward and Roger (1997), so results should agree closely with KRmodcomp() in the R package pbkrtest applied to an lmer fit, and with SAS PROC MIXED using DDFM=KENWARDROGER.

Small differences can survive. REML optimizers stop at slightly different places, and programs parameterize the covariance structure differently. Kenward and Roger published a revised approximation in 2009 for covariance structures that are nonlinear in their parameters. Compound symmetry is linear, so the 1997 and 2009 versions agree for the models Prism fits.

References

1.Kenward MG, Roger JH. Small sample inference for fixed effects from restricted maximum likelihood. Biometrics 1997;53(3):983-997.

2.Halekoh U, Højsgaard S. A Kenward-Roger approximation and parametric bootstrap methods for tests in linear mixed models: the R package pbkrtest. Journal of Statistical Software 2014;59(9):1-30.

3.Kenward MG, Roger JH. An improved approximation to the precision of fixed effects from restricted maximum likelihood. Computational Statistics and Data Analysis 2009;53(7):2583-2595.