Mean-field Variational Bayes for Sparse Probit Regression

TL;DR

This paper introduces a fast mean-field variational Bayes method for sparse probit regression with spike-and-slab priors, enabling efficient variable selection and prediction in high-dimensional settings.

Contribution

It develops a closed-form, coordinate ascent variational inference algorithm that outperforms MCMC in speed while maintaining accuracy.

Findings

Successfully identifies important variables in simulations and real data.

Achieves orders of magnitude faster computation than MCMC.

Provides comparable accuracy to traditional Bayesian methods.

Abstract

We consider Bayesian variable selection for binary outcomes under a probit link with a spike-and-slab prior on the regression coefficients. Motivated by the computational challenges encountered by Markov chain Monte Carlo (MCMC) samplers in high-dimensional regimes, we develop a mean-field variational Bayes approximation in which all variational factors admit closed-form updates, and the evidence lower bound is available in closed form. This, in turn, allows the development of an efficient coordinate ascent variational inference algorithm to find the optimal values of the variational parameters. The approach produces posterior inclusion probabilities and parameter estimates, enabling interpretable selection and prediction within a single framework. As shown in both simulated and real data applications, the proposed method successfully identifies the important variables and is orders of magnitude faster than MCMC, while maintaining comparable accuracy.