Posterior Asymptotic Normality for an Individual Coordinate in High-dimensional Linear Regression

TL;DR

This paper demonstrates that in high-dimensional linear regression, the posterior distribution of an individual coefficient becomes asymptotically normal, enabling Bayesian credible intervals to align with traditional confidence intervals.

Contribution

The paper introduces a prior ensuring the marginal posterior of a coordinate is asymptotically normal, matching efficient estimators, and compares methods for obtaining such estimators.

Findings

Posterior distribution of a coordinate is asymptotically normal.

Bayesian credible intervals can match confidence intervals.

Comparison of two estimation procedures.

Abstract

We consider the sparse high-dimensional linear regression model $Y=Xb+\epsilon$ where $b$ is a sparse vector. For the Bayesian approach to this problem, many authors have considered the behavior of the posterior distribution when, in truth, $Y=X\beta+\epsilon$ for some given $\beta$. There have been numerous results about the rate at which the posterior distribution concentrates around $\beta$, but few results about the shape of that posterior distribution. We propose a prior distribution for $b$ such that the marginal posterior distribution of an individual coordinate $b_i$ is asymptotically normal centered around an asymptotically efficient estimator, under the truth. Such a result gives Bayesian credible intervals that match with the confidence intervals obtained from an asymptotically efficient estimator for $b_i$. We also discuss ways of obtaining such asymptotically efficient estimators on individual coordinates. We compare the two-step procedure proposed by Zhang and Zhang (2014) and a one-step modified penalization method.