Bayesian Sparse Global-Local Shrinkage Regression for Selection of Grouped Variables

TL;DR

This paper introduces a Bayesian grouped shrinkage model with hierarchical priors for variable selection in high-dimensional linear models, extending existing methods to handle complex group structures and improving group selection accuracy.

Contribution

It develops a Bayesian grouped model with continuous global-local priors for complex hierarchies, and adapts decoupled shrinkage and selection for group variable selection.

Findings

Effective identification of active and inactive groups in simulations

Improved prediction accuracy over existing Bayesian methods

Promising results in real data analysis

Abstract

Most estimates for penalised linear regression can be viewed as posterior modes for an appropriate choice of prior distribution. Bayesian shrinkage methods, particularly the horseshoe estimator, have recently attracted a great deal of attention in the problem of estimating sparse, high-dimensional linear models. This paper extends these ideas, and presents a Bayesian grouped model with continuous global-local shrinkage priors to handle complex group hierarchies that include overlapping and multilevel group structures. As the posterior mean is never a sparse estimate of the linear model coefficients, we extend the recently proposed decoupled shrinkage and selection (DSS) technique to the problem of selecting groups of variables from posterior samples. To choose a final, sparse model, we also adapt generalised information criteria approaches to the DSS framework. To ensure that sparse groups, in which only a few predictors are active, can be effectively identified, we provide an alternative degrees of freedom estimator for sparse Bayesian linear models that takes into account the effects of shrinkage on the model coefficients. Simulations and real data analysis using our proposed method show promising performance in terms of correct identification of active and inactive groups, and prediction, in comparison with a Bayesian grouped slab-and-spike approach.