Gradient Boosting for Linear Mixed Models

TL;DR

This paper introduces a new gradient boosting algorithm tailored for linear mixed models, addressing biases and convergence issues in existing methods, and providing likelihood-based estimation of random effects.

Contribution

It proposes an unbiased boosting algorithm that explicitly accounts for random effects, improving estimation accuracy and convergence in mixed models.

Findings

The new algorithm reduces bias in random effects estimates.

It demonstrates improved convergence over existing methods.

Simulation and data examples validate the approach.

Abstract

Gradient boosting from the field of statistical learning is widely known as a powerful framework for estimation and selection of predictor effects in various regression models by adapting concepts from classification theory. Current boosting approaches also offer methods accounting for random effects and thus enable prediction of mixed models for longitudinal and clustered data. However, these approaches include several flaws resulting in unbalanced effect selection with falsely induced shrinkage and a low convergence rate on the one hand and biased estimates of the random effects on the other hand. We therefore propose a new boosting algorithm which explicitly accounts for the random structure by excluding it from the selection procedure, properly correcting the random effects estimates and in addition providing likelihood-based estimation of the random effects variance structure. The new algorithm offers an organic and unbiased fitting approach, which is shown via simulations and data examples.