Bayesian linear regression with sparse priors

TL;DR

This paper analyzes Bayesian linear regression with sparse priors, demonstrating optimal recovery, prediction, and model selection in high-dimensional settings, along with credible set construction for uncertainty quantification.

Contribution

It provides theoretical guarantees for posterior contraction, model selection, and credible sets under sparsity and design matrix conditions.

Findings

Posterior contracts at the optimal rate for sparse recovery

Achieves optimal prediction of responses

Correctly identifies significant coefficients

Abstract

We study full Bayesian procedures for high-dimensional linear regression under sparsity constraints. The prior is a mixture of point masses at zero and continuous distributions. Under compatibility conditions on the design matrix, the posterior distribution is shown to contract at the optimal rate for recovery of the unknown sparse vector, and to give optimal prediction of the response vector. It is also shown to select the correct sparse model, or at least the coefficients that are significantly different from zero. The asymptotic shape of the posterior distribution is characterized and employed to the construction and study of credible sets for uncertainty quantification.