Algorithms for Kullback-Leibler Approximation of Probability Measures in Infinite Dimensions

TL;DR

This paper develops algorithms to approximate complex probability measures in infinite-dimensional spaces using Gaussian measures by minimizing Kullback-Leibler divergence, with applications to PDE inverse problems and diffusion processes.

Contribution

It introduces a novel computational approach for Gaussian approximation in infinite dimensions, including covariance parameterizations and applications to inverse problems and diffusion.

Findings

Effective Gaussian approximations for complex measures

Improved pCN-MCMC methods for high-dimensional settings

Robustness to small observational noise and temperature variations

Abstract

In this paper we study algorithms to find a Gaussian approximation to a target measure defined on a Hilbert space of functions; the target measure itself is defined via its density with respect to a reference Gaussian measure. We employ the Kullback-Leibler divergence as a distance and find the best Gaussian approximation by minimizing this distance. It then follows that the approximate Gaussian must be equivalent to the Gaussian reference measure, defining a natural function space setting for the underlying calculus of variations problem. We introduce a computational algorithm which is well-adapted to the required minimization, seeking to find the mean as a function, and parameterizing the covariance in two different ways: through low rank perturbations of the reference covariance; and through Schr\"odinger potential perturbations of the inverse reference covariance. Two applications are shown: to a nonlinear inverse problem in elliptic PDEs, and to a conditioned diffusion process. We also show how the Gaussian approximations we obtain may be used to produce improved pCN-MCMC methods which are not only well-adapted to the high-dimensional setting, but also behave well with respect to small observational noise (resp. small temperatures) in the inverse problem (resp. conditioned diffusion).