Gaussian Approximation of General Nonparametric Posterior Distributions

TL;DR

This paper proves that in a broad class of Bayesian nonparametric models, the posterior distribution can be approximated by a Gaussian process, facilitating understanding of Bayesian procedures' asymptotic behavior.

Contribution

It establishes a general Gaussian approximation theorem for nonparametric posteriors that does not depend on conjugacy and applies to a wide range of models including exponential families.

Findings

Posterior distributions can be asymptotically approximated by Gaussian processes.

The approximation holds for both efficient and inefficient estimators.

Bayesian credible regions exhibit desirable frequentist properties under the approximation.

Abstract

In a general class of Bayesian nonparametric models, we prove that the posterior distribution can be asymptotically approximated by a Gaussian process. Our results apply to nonparametric exponential family that contains both Gaussian and non-Gaussian regression, and also hold for both efficient (root-n) and inefficient (non root-n) estimation. Our general approximation theorem does not rely on posterior conjugacy, and can be verified in a class of Gaussian process priors that has a smoothing spline interpretation [59, 44]. In particular, the limiting posterior measure becomes prior-free under a Bayesian version of "under-smoothing" condition. Finally, we apply our approximation theorem to examine the asymptotic frequentist properties of Bayesian procedures such as credible regions and credible intervals.