Improved dimension dependence in the Bernstein von Mises Theorem via a new Laplace approximation bound

TL;DR

This paper improves the understanding of high-dimensional Bayesian asymptotics by establishing that the Bernstein-von Mises theorem holds under milder conditions, specifically when the sample size grows faster than the square of the dimension, using a new Laplace approximation bound.

Contribution

The paper introduces a new explicit Laplace approximation bound that reduces the sample size requirement for the Bernstein-von Mises theorem from cubic to quadratic in the dimension.

Findings

BvM holds when n ≫ d² in generalized linear models and multinomial data.

Improves previous condition n ≫ d³ for posterior normality.

Provides nonasymptotic, high-probability bounds for the BvM.

Abstract

The Bernstein-von Mises theorem (BvM) gives conditions under which the posterior distribution of a parameter $\theta\in\Theta\subseteq\mathbb R^d$ based on $n$ independent samples is asymptotically normal. In the high-dimensional regime, a key question is to determine the growth rate of $d$ with $n$ required for the BvM to hold. We show that up to a model-dependent coefficient, $n\gg d^2$ suffices for the BvM to hold in two settings: arbitrary generalized linear models, which include exponential families as a special case, and multinomial data, in which the parameter of interest is an unknown probability mass functions on $d+1$ states. Our results improve on the tightest previously known condition for posterior asymptotic normality, $n\gg d^3$. Our statements of the BvM are nonasymptotic, taking the form of explicit high-probability bounds. To prove the BvM, we derive a new simple and explicit bound on the total variation distance between a measure $\pi\propto e^{-nf}$ on $\Theta\subseteq\mathbb R^d$ and its Laplace approximation.