Bayesian Boosting for Linear Mixed Models

TL;DR

This paper introduces BayesBoost, a novel method combining boosting and Bayesian inference for linear mixed models, enabling uncertainty estimation and improved covariate selection in high-dimensional data analysis.

Contribution

The paper presents a new inference technique that integrates boosting with Bayesian methods for linear mixed models, addressing uncertainty estimation and covariate selection challenges.

Findings

Effective uncertainty estimation for random effects demonstrated.

Improved covariate selection using cAIC-based criteria.

Method validated through simulation and neurophysiology data.

Abstract

Boosting methods are widely used in statistical learning to deal with high-dimensional data due to their variable selection feature. However, those methods lack straightforward ways to construct estimators for the precision of the parameters such as variance or confidence interval, which can be achieved by conventional statistical methods like Bayesian inference. In this paper, we propose a new inference method "BayesBoost" that combines boosting and Bayesian for linear mixed models to make the uncertainty estimation for the random effects possible on the one hand. On the other hand, the new method overcomes the shortcomings of Bayesian inference in giving precise and unambiguous guidelines for the selection of covariates by benefiting from boosting techniques. The implementation of Bayesian inference leads to the randomness of model selection criteria like the conditional AIC (cAIC), so we also propose a cAIC-based model selection criteria that focus on the stabilized regions instead of the global minimum. The effectiveness of the new approach can be observed via simulation and in a data example from the field of neurophysiology focussing on the mechanisms in the brain while listening to unpleasant sounds.