Subset selection for linear mixed models

TL;DR

This paper presents a Bayesian decision analysis approach for subset selection in linear mixed models, effectively handling structured dependence and providing stable, near-optimal variable subsets with uncertainty quantification.

Contribution

It introduces a novel Bayesian decision framework for subset selection in LMMs that emphasizes multiple near-optimal subsets over a single best subset.

Findings

Demonstrates superior prediction and estimation on simulated data.

Provides scalable algorithms for subset search.

Shows effective variable importance metrics.

Abstract

Linear mixed models (LMMs) are instrumental for regression analysis with structured dependence, such as grouped, clustered, or multilevel data. However, selection among the covariates--while accounting for this structured dependence--remains a challenge. We introduce a Bayesian decision analysis for subset selection with LMMs. Using a Mahalanobis loss function that incorporates the structured dependence, we derive optimal linear coefficients for (i) any given subset of variables and (ii) all subsets of variables that satisfy a cardinality constraint. Crucially, these estimates inherit shrinkage or regularization and uncertainty quantification from the underlying Bayesian model, and apply for any well-specified Bayesian LMM. More broadly, our decision analysis strategy deemphasizes the role of a single "best" subset, which is often unstable and limited in its information content, and instead favors a collection of near-optimal subsets. This collection is summarized by key member subsets and variable-specific importance metrics. Customized subset search and out-of-sample approximation algorithms are provided for more scalable computing. These tools are applied to simulated data and a longitudinal physical activity dataset, and demonstrate excellent prediction, estimation, and selection ability.