Dimension free non-asymptotic bounds on the accuracy of high dimensional Laplace approximation

TL;DR

This paper provides non-asymptotic, dimension-free bounds on the accuracy of Laplace approximation in high-dimensional settings, emphasizing the role of effective dimension and addressing practical algorithmic applications.

Contribution

It extends classical Laplace approximation results to a non-asymptotic, dimension-free framework applicable to high-dimensional problems, incorporating prior influence and inexact parameter use.

Findings

Explicit bounds on Gaussian approximation quality in total variation distance.

Effective dimension captures data-prior interplay, controlling approximation accuracy.

Application to Bayesian optimization and nonlinear inverse problems.

Abstract

This note attempts to revisit the classical results on Laplace approximation in a modern non-asymptotic and dimension free form. Such an extension is motivated by applications to high dimensional statistical and optimization problems. The established results provide explicit non-asymptotic bounds on the quality of a Gaussian approximation of the posterior distribution in total variation distance in terms of the so called \emph{effective dimension} \( p_G \). This value is defined as interplay between information contained in the data and in the prior distribution. In the contrary to prominent Bernstein - von Mises results, the impact of the prior is not negligible and it allows to keep the effective dimension small or moderate even if the true parameter dimension is huge or infinite. We also address the issue of using a Gaussian approximation with inexact parameters with the focus on replacing the Maximum a Posteriori (MAP) value by the posterior mean and design the algorithm of Bayesian optimization based on Laplace iterations. The results are specified to the case of nonlinear inverse problem.