Restricted maximum likelihood estimation in generalized linear mixed models

TL;DR

This paper reviews methods for extending restricted maximum likelihood (REML) estimation to generalized linear mixed models, comparing their performance and advocating for wider adoption in practice.

Contribution

It categorizes four main approaches to REML in generalized linear mixed models and compares their finite sample performance through a comprehensive numerical study.

Findings

All approaches similarly reduce bias in variance estimates.

REML methods perform well in finite samples for binary and count data.

Choice of REML method should be based on software availability and ease of use.

Abstract

Restricted maximum likelihood (REML) estimation is a widely accepted and frequently used method for fitting linear mixed models, with its principal advantage being that it produces less biased estimates of the variance components. However, the concept of REML does not immediately generalize to the setting of non-normally distributed responses, and it is not always clear the extent to which, either asymptotically or in finite samples, such generalizations reduce the bias of variance component estimates compared to standard unrestricted maximum likelihood estimation. In this article, we review various attempts that have been made over the past four decades to extend REML estimation in generalized linear mixed models. We establish four major classes of approaches, namely approximate linearization, integrated likelihood, modified profile likelihoods, and direct bias correction of the score function, and show that while these four classes may have differing motivations and derivations, they often arrive at a similar if not the same REML estimate. We compare the finite sample performance of these four classes, along with methods for REML estimation in hierarchical generalized linear models, through a numerical study involving binary and count data, with results demonstrating that all approaches perform similarly well reducing the finite sample size bias of variance components. Overall, we believe REML estimation should more widely adopted by practitioners using generalized linear mixed models, and that the exact choice of which REML approach to use should, at this point in time, be driven by software availability and ease of implementation.