Bayesian Compressed Mixed-Effects Models

TL;DR

This paper introduces the compressed mixed-effects (CME) model, a Bayesian approach that reduces computational complexity in high-dimensional linear mixed-effects models through covariance matrix compression, enabling efficient inference and accurate prediction.

Contribution

The paper proposes the CME model that employs low-dimensional covariance approximations and a global-local shrinkage prior, improving Bayesian inference in high-dimensional mixed-effects models.

Findings

CME achieves superior predictive accuracy over existing methods.

CME provides better interval coverage in simulations.

CME effectively selects fixed effects in diverse settings.

Abstract

Penalized likelihood and quasi-likelihood methods dominate inference in high-dimensional linear mixed-effects models. Sampling-based Bayesian inference is less explored due to the computational bottlenecks introduced by the random effects covariance matrix. To address this gap, we propose the compressed mixed-effects (CME) model, which defines a quasi-likelihood using low-dimensional covariance parameters obtained via random projections of the random effects covariance. This dimension reduction, combined with a global-local shrinkage prior on the fixed effects, yields an efficient collapsed Gibbs sampler for prediction and fixed effects selection. Theoretically, when the compression dimension grows slowly relative to the number of fixed effects and observations, the Bayes risk for prediction is asymptotically negligible, ensuring accurate prediction using the CME model. Empirically, the CME model outperforms existing approaches in terms of predictive accuracy, interval coverage, and fixed-effects selection across varied simulation settings and a real-world dataset.