A Variational Inference method for Bayesian variable selection

TL;DR

This paper introduces a variational inference approach for Bayesian variable selection using spike-and-slab priors, providing an efficient alternative to MCMC with proven consistency and asymptotic properties.

Contribution

It develops a collapsed variational inference method for Bayesian variable selection with spike-and-slab priors, extending previous work with theoretical guarantees.

Findings

Efficient variational inference algorithm for variable selection

Proven consistency and asymptotic properties of the estimator

Comparison shows improved computational performance over MCMC

Abstract

Variable selection is a classic problem in statistics. In this paper, we consider a Bayes variable selection problem based on spike-and-slab prior with mixed normal distribution proposed by Ro\v{c}kov\'a and George (2014). Motivated by Ormerod and You (2017, 2023), we use the variational inference and collapsed variational inference method to solve the Bayesian problem instead of MCMC. Like Ormerod and You (2017, 2023), we also explain how the sparsity estimator is induced, and under certain mild assumptions, we also prove the consistent and asymptotic results.