Bayesian High-dimensional Semi-parametric Inference beyond sub-Gaussian Errors

TL;DR

This paper develops Bayesian methods for high-dimensional sparse linear regression with unknown symmetric errors, achieving near-optimal convergence rates and establishing asymptotic normality of the posterior for coefficients.

Contribution

It introduces a Bayesian framework that handles non-sub-Gaussian errors, providing convergence rates and a semi-parametric Bernstein-von Mises theorem for inference.

Findings

Posterior convergence rates are nearly optimal and adaptive.

Semi-parametric BvM theorem characterizes the asymptotic shape of the posterior.

Strong model selection consistency under sub-Gaussian score functions.

Abstract

We consider a sparse linear regression model with unknown symmetric error under the high-dimensional setting. The true error distribution is assumed to belong to the locally $\beta$-H\"{o}lder class with an exponentially decreasing tail, which does not need to be sub-Gaussian. We obtain posterior convergence rates of the regression coefficient and the error density, which are nearly optimal and adaptive to the unknown sparsity level. Furthermore, we derive the semi-parametric Bernstein-von Mises (BvM) theorem to characterize asymptotic shape of the marginal posterior for regression coefficients. Under the sub-Gaussianity assumption on the true score function, strong model selection consistency for regression coefficients are also obtained, which eventually asserts the frequentist's validity of credible sets.