Fisher Scoring for crossed factor Linear Mixed Models

TL;DR

This paper extends Fisher Scoring algorithms to multi-factored Linear Mixed Models, demonstrating their efficiency and correctness through simulations and real data, and introduces a new method for degrees of freedom estimation with improved bias and variance.

Contribution

It develops new Fisher Scoring variants for crossed-factor LMMs and proposes a non-iterative Satterthwaite degrees of freedom estimation method.

Findings

Four Fisher Scoring variants are computationally efficient and correct.

The new degrees of freedom method has lower bias and variance.

Simulation and real data validate the methods' effectiveness.

Abstract

The analysis of longitudinal, heterogeneous or unbalanced clustered data is of primary importance to a wide range of applications. The Linear Mixed Model (LMM) is a popular and flexible extension of the linear model specifically designed for such purposes. Historically, a large proportion of material published on the LMM concerns the application of popular numerical optimization algorithms, such as Newton-Raphson, Fisher Scoring and Expectation Maximization to single-factor LMMs (i.e. LMMs that only contain one "factor" by which observations are grouped). However, in recent years, the focus of the LMM literature has moved towards the development of estimation and inference methods for more complex, multi-factored designs. In this paper, we present and derive new expressions for the extension of an algorithm classically used for single-factor LMM parameter estimation, Fisher Scoring, to multiple, crossed-factor designs. Through simulation and real data examples, we compare five variants of the Fisher Scoring algorithm with one another, as well as against a baseline established by the R package lmer, and find evidence of correctness and strong computational efficiency for four of the five proposed approaches. Additionally, we provide a new method for LMM Satterthwaite degrees of freedom estimation based on analytical results, which does not require iterative gradient estimation. Via simulation, we find that this approach produces estimates with both lower bias and lower variance than the existing methods.