Motivating REML via Prediction-Error Covariances in EM Updates for Linear Mixed Models

TL;DR

This paper provides a computational perspective on REML estimation in linear mixed models by framing it within an EM algorithm, highlighting the role of prediction-error covariances in variance updates.

Contribution

It introduces a novel EM-based interpretation of REML that clarifies the use of prediction-error covariances, with accessible R code illustrating the approach.

Findings

REML differs from ML in variance updates through prediction-error covariances.

The EM algorithm framework makes the REML estimation process more transparent.

The R code reproduces standard lme4 ML and REML fits, facilitating understanding.

Abstract

We present a computational motivation for restricted maximum likelihood (REML) estimation in linear mixed models using an expectation--maximization (EM) algorithm. At each iteration, maximum likelihood (ML) and REML solve the same mixed-model equations for the best linear unbiased estimator (BLUE) of the fixed effects and the best linear unbiased predictor (BLUP) of the random effects. They differ only in the trace adjustments used in the variance-component updates: ML uses conditional covariances of the random effects given the data, whereas REML uses prediction-error covariances from Henderson's C-matrix, reflecting uncertainty from estimating the fixed effects. Short R code makes this switch explicit, exposes the key matrices for classroom inspection, and reproduces lme4 ML and REML fits.