Conjugate gradient methods for high-dimensional GLMMs

TL;DR

This paper investigates the use of conjugate gradient methods for efficiently solving high-dimensional generalized linear mixed models (GLMMs), demonstrating linear scaling of computational cost with data size and analyzing spectral properties of the involved matrices.

Contribution

It introduces a novel analysis combining spectral theory and random graph insights to justify CG methods for high-dimensional GLMMs, highlighting their efficiency and limitations.

Findings

CG methods achieve fixed approximation error with linear cost scaling

Cholesky factorization is dense even for sparse precision matrices

Numerical experiments confirm theoretical efficiency and identify challenging structures

Abstract

Generalized linear mixed models (GLMMs) are a widely used tool in statistical analysis. The main bottleneck of many computational approaches lies in the inversion of the high dimensional precision matrices associated with the random effects. Such matrices are typically sparse; however, the sparsity pattern resembles a multi partite random graph, which does not lend itself well to default sparse linear algebra techniques. Notably, we show that, for typical GLMMs, the Cholesky factor is dense even when the original precision is sparse. We thus turn to approximate iterative techniques, in particular to the conjugate gradient (CG) method. We combine a detailed analysis of the spectrum of said precision matrices with results from random graph theory to show that CG-based methods applied to high-dimensional GLMMs typically achieve a fixed approximation error with a total cost that scales linearly with the number of parameters and observations. Numerical illustrations with both real and simulated data confirm the theoretical findings, while at the same time illustrating situations, such as nested structures, where CG-based methods struggle.