An approach to large-scale Quasi-Bayesian inference with spike-and-slab priors

TL;DR

This paper introduces a scalable Bayesian inference framework using spike-and-slab priors for high-dimensional models, with efficient MCMC algorithms and theoretical guarantees for variational approximations.

Contribution

It develops a general quasi-Bayesian framework with scalable algorithms and theoretical analysis for high-dimensional inference using spike-and-slab priors.

Findings

Efficient MCMC algorithms for quasi-posterior sampling.

Convergence results for large-scale quasi-posterior distributions.

Theoretical guarantees for variational approximation accuracy.

Abstract

We propose a general framework using spike-and-slab prior distributions to aid with the development of high-dimensional Bayesian inference. Our framework allows inference with a general quasi-likelihood function. We show that highly efficient and scalable Markov Chain Monte Carlo (MCMC) algorithms can be easily constructed to sample from the resulting quasi-posterior distributions. We study the large scale behavior of the resulting quasi-posterior distributions as the dimension of the parameter space grows, and we establish several convergence results. In large-scale applications where computational speed is important, variational approximation methods are often used to approximate posterior distributions. We show that the contraction behaviors of the quasi-posterior distributions can be exploited to provide theoretical guarantees for their variational approximations. We illustrate the theory with some simulation results from Gaussian graphical models, and sparse principal component analysis.