Accuracy of Gaussian approximation in nonparametric Bernstein -- von Mises Theorem

TL;DR

This paper investigates the accuracy of Gaussian approximation in nonparametric Bernstein--von Mises theorems, providing finite sample bounds and showing that credible sets can be approximated by Gaussian distributions with high precision.

Contribution

It introduces a non-asymptotic approach to analyze the posterior distribution for models with Gaussian priors, deriving bounds and conditions for Gaussian approximation accuracy.

Findings

Finite sample bounds on posterior contraction based on effective dimension

Gaussian approximation of the posterior is accurate up to order n^{-1}

Posterior distribution closely mimics the penalized maximum likelihood estimator

Abstract

The prominent Bernstein -- von Mises (BvM) result claims that the posterior distribution after centering by the efficient estimator and standardizing by the square root of the total Fisher information is nearly standard normal. In particular, the prior completely washes out from the asymptotic posterior distribution. This fact is fundamental and justifies the Bayes approach from the frequentist viewpoint. In the nonparametric setup the situation changes dramatically and the impact of prior becomes essential even for the contraction of the posterior; see [vdV2008], [Bo2011], [CaNi2013,CaNi2014] for different models like Gaussian regression or i.i.d. model in different weak topologies. This paper offers another non-asymptotic approach to studying the behavior of the posterior for a special but rather popular and useful class of statistical models and for Gaussian priors. First we derive tight finite sample bounds on posterior contraction in terms of the so called effective dimension of the parameter space. Our main results describe the accuracy of Gaussian approximation of the posterior. In particular, we show that restricting to the class of all centrally symmetric credible sets around pMLE allows to get Gaussian approximation up to order (n^{-1}). We also show that the posterior distribution mimics well the distribution of the penalized maximum likelihood estimator (pMLE) and reduce the question of reliability of credible sets to consistency of the pMLE-based confidence sets. The obtained results are specified for nonparametric log-density estimation and generalized regression.