Variational approximate penalized credible regions for Bayesian grouped regression

TL;DR

This paper introduces a fast Bayesian grouped regression method that combines variational inference with penalized credible regions to achieve accurate, sparse, and computationally efficient variable selection and prediction in high-dimensional settings.

Contribution

It develops a novel variational approximation framework for penalized credible regions that enables scalable, accurate, and unambiguous grouped variable selection in Bayesian high-dimensional regression.

Findings

Outperforms existing methods in variable selection and prediction accuracy.

Achieves significant reductions in computation time.

Provides theoretical guarantees for consistency in high-dimensional regimes.

Abstract

We develop a fast and accurate grouped penalized credible region approach for variable selection and prediction in Bayesian high-dimensional linear regression. Most existing Bayesian methods either are subject to high computational costs due to long Markov Chain Monte Carlo runs or yield ambiguous variable selection results due to non-sparse solution output. The penalized credible region framework yields sparse post-processed estimates that facilitates unambiguous grouped variable selection. High estimation accuracy is achieved by shrinking noise from unimportant groups using a grouped global-local shrinkage prior. To ensure computational scalability, we approximate posterior summaries using coordinate ascent variational inference and recast the penalized credible region framework as a convex optimization problem that admits efficient computations. We prove that the resultant post-processed estimators are both parameter-consistent and variable selection consistent in high-dimensional settings. Theory is developed to justify running the coordinate ascent algorithm for at least two cycles. Through extensive simulations, we demonstrate that our proposed method outperforms state-of-the-art methods in grouped variable selection, prediction, and computation time for several common models including ANOVA and nonparametric varying coefficient models.