On frequentist coverage errors of Bayesian credible sets in moderately high dimensions

TL;DR

This paper investigates the accuracy of Bayesian credible sets in high-dimensional linear regression, providing finite-sample bounds on coverage errors and demonstrating the advantages of Bayesian bands over traditional confidence bands.

Contribution

It introduces a novel Berry--Esseen bound for quasi-posterior distributions and applies it to quantify coverage errors in various high-dimensional and nonparametric models.

Findings

Coverage errors decay polynomially with sample size.

Bayesian credible bands outperform confidence bands in nonparametric models.

Finite sample bounds are derived for high-dimensional settings.

Abstract

In this paper, we study frequentist coverage errors of Bayesian credible sets for an approximately linear regression model with (moderately) high dimensional regressors, where the dimension of the regressors may increase with but is smaller than the sample size. Specifically, we consider quasi-Bayesian inference on the slope vector under the quasi-likelihood with Gaussian error distribution. Under this setup, we derive finite sample bounds on frequentist coverage errors of Bayesian credible rectangles. Derivation of those bounds builds on a novel Berry--Esseen type bound on quasi-posterior distributions and recent results on high-dimensional CLT on hyperrectangles. We use this general result to quantify coverage errors of Castillo--Nickl and $L^{\infty}$-credible bands for Gaussian white noise models, linear inverse problems, and (possibly non-Gaussian) nonparametric regression models. In particular, we show that Bayesian credible bands for those nonparametric models have coverage errors decaying polynomially fast in the sample size, implying advantages of Bayesian credible bands over confidence bands based on extreme value theory.