Gradient Boosted Mixed Models: Flexible Joint Estimation of Mean and Variance Components for Clustered Data

TL;DR

The paper introduces GBMixed, a novel boosting framework that jointly estimates mean and variance components in clustered data, allowing flexible, nonparametric modeling of heteroscedasticity and random effects with uncertainty quantification.

Contribution

It extends gradient boosting to jointly estimate mean and variance in mixed models, accommodating covariate-dependent effects and heteroscedasticity, which is a novel approach.

Findings

Accurately recovers variance components in simulations

Provides well-calibrated prediction intervals

Improves predictive accuracy over traditional models

Abstract

Linear mixed models are widely used for clustered data, but their reliance on parametric forms limits flexibility in complex and high-dimensional settings. In contrast, gradient boosting methods achieve high predictive accuracy through nonparametric estimation, but do not accommodate clustered data structures or provide uncertainty quantification. We introduce Gradient Boosted Mixed Models (GBMixed), a framework and algorithm that extends boosting to jointly estimate mean and variance components via likelihood-based gradients. In addition to nonparametric mean estimation, the method models both random effects and residual variances as potentially covariate-dependent functions using flexible base learners such as regression trees or splines, enabling nonparametric estimation while maintaining interpretability. Simulations and real-world applications demonstrate accurate recovery of variance components, calibrated prediction intervals, and improved predictive accuracy relative to standard linear mixed models and nonparametric methods. GBMixed provides heteroscedastic uncertainty quantification and introduces boosting for heterogeneous random effects. This enables covariate-dependent shrinkage for cluster-specific predictions to adapt between population and cluster-level data. Under standard causal assumptions, the framework enables estimation of heterogeneous treatment effects with reliable uncertainty quantification.