Scalable Krylov Subspace Methods for Generalized Mixed-Effects Models with Crossed Random Effects

TL;DR

This paper introduces Krylov subspace methods to efficiently compute generalized mixed-effects models with crossed random effects, significantly improving speed and stability over traditional Cholesky-based approaches.

Contribution

It develops scalable Krylov subspace algorithms with theoretical analysis and demonstrates their effectiveness on real and simulated data.

Findings

Speedups of up to 10,000 times compared to Cholesky methods

Methods are numerically more stable

Effective for high-dimensional crossed random effects

Abstract

Mixed-effects models are widely used to model data with hierarchical grouping structures and high-cardinality categorical predictor variables. However, for high-dimensional crossed random effects, current standard computations relying on Cholesky decompositions can become prohibitively slow. In this work, we present Krylov subspace-based methods that address existing computational bottlenecks, and we analyze them both theoretically and empirically. In particular, we derive new results on the convergence and accuracy of the preconditioned stochastic Lanczos quadrature and conjugate gradient methods for mixed-effects models, and we develop scalable methods for calculating predictive variances. In experiments with simulated and real-world data, the proposed methods yield speedups by factors of up to about 10,000 and are numerically more stable than Cholesky-based computations.