Adaptive posterior concentration rates for sparse high-dimensional linear regression with random design and unknown error variance

TL;DR

This paper studies the behavior of the Bayesian posterior in high-dimensional sparse linear regression with random design and unknown variance, showing it adapts to sparsity and addresses model misspecification.

Contribution

It introduces adaptive posterior concentration results for sparse high-dimensional regression with unknown variance, extending analysis to model misspecification via fractional posteriors.

Findings

Posterior concentrates at optimal rates under sparsity.

Demonstrates adaptiveness to unknown sparsity levels.

Addresses model misspecification with fractional posterior.

Abstract

This paper investigates sparse high-dimensional linear regression, particularly examining the properties of the posterior under conditions of random design and unknown error variance. We provide consistency results for the posterior and analyze its concentration rates, demonstrating adaptiveness to the unknown sparsity level of the regression coefficient vector. Furthermore, we extend our investigation to establish concentration outcomes for parameter estimation using specific distance measures. These findings are in line with recent discoveries in frequentist studies. Additionally, by employing techniques to address model misspecification through a fractional posterior, we broaden our analysis through oracle inequalities to encompass the critical aspect of model misspecification for the regular posterior. Our novel findings are demonstrated using two different types of sparsity priors: a shrinkage prior and a spike-and-slab prior.