A Variational Spike-and-Slab Approach for Group Variable Selection

TL;DR

This paper proposes a flexible spike-and-slab Bayesian approach with a novel variational inference algorithm for high-dimensional grouped variable selection, demonstrating improved efficiency and accuracy over existing methods.

Contribution

It introduces a new class of spike-and-slab priors and a coordinate-ascent variational inference algorithm with parameter expansion, extending to additive models and providing theoretical contraction rates.

Findings

Outperforms existing methods in variable selection accuracy

Achieves computational efficiency through parameter expansion

Demonstrates superior parameter estimation in simulations and real data

Abstract

We introduce a class of generic spike-and-slab priors for high-dimensional linear regression with grouped variables and present a Coordinate-ascent Variational Inference (CAVI) algorithm for obtaining an optimal variational Bayes approximation. Using parameter expansion for a specific, yet comprehensive, family of slab distributions, we obtain a further gain in computational efficiency. The method can be easily extended to fitting additive models. Theoretically, we present general conditions on the generic spike-and-slab priors that enable us to derive the contraction rates for both the true posterior and the VB posterior for linear regression and additive models, of which some previous theoretical results can be viewed as special cases. Our simulation studies and real data application demonstrate that the proposed method is superior to existing methods in both variable selection and parameter estimation. Our algorithm is implemented in the R package GVSSB.