From Estimation to Sampling for Bayesian Linear Regression with Spike-and-Slab Prior

TL;DR

This paper develops efficient sampling algorithms for Bayesian linear regression with spike-and-slab priors, improving inference on sparse signals especially in challenging data matrix conditions.

Contribution

It introduces and analyzes Gibbs sampling and Stochastic Localization algorithms leveraging posterior contraction, with a focus on a Gaussian spike-and-slab quasi-likelihood.

Findings

Stochastic Localization performs well on poorly designed data matrices.

Both algorithms enable valid inference on sparse signals.

The proposed methods are computationally and statistically advantageous.

Abstract

We consider Bayesian linear regression with sparsity-inducing prior and design efficient sampling algorithms leveraging posterior contraction properties. A quasi-likelihood with Gaussian spike-and-slab (that is favorable both statistically and computationally) is investigated and two algorithms based on Gibbs sampling and Stochastic Localization are analyzed, both under the same (quite natural) statistical assumptions that also enable valid inference on the sparse planted signal. The benefit of the Stochastic Localization sampler is particularly prominent for data matrix that is not well-designed.