Geometric Gaussian Approximations of Probability Distributions

TL;DR

This paper investigates the use of geometric Gaussian approximations, which involve transforming Gaussian distributions through diffeomorphisms or Riemannian maps, demonstrating their universality and exploring their applicability to various probability distributions.

Contribution

It introduces and analyzes the expressivity of geometric Gaussian approximations, showing they can universally approximate any distribution and examining the potential for common transformations across distribution families.

Findings

Geometric Gaussian approximations are shown to be universal.

The relationship between diffeomorphism-based and Riemannian exponential map-based approximations is explored.

Discussion on the existence of common transformations for families of distributions.

Abstract

Approximating complex probability distributions, such as Bayesian posterior distributions, is of central interest in many applications. We study the expressivity of geometric Gaussian approximations. These consist of approximations by Gaussian pushforwards through diffeomorphisms or Riemannian exponential maps. We first review these two different kinds of geometric Gaussian approximations. Then we explore their relationship to one another. We further provide a constructive proof that such geometric Gaussian approximations are universal, in that they can capture any probability distribution. Finally, we discuss whether, given a family of probability distributions, a common diffeomorphism can be found to obtain uniformly high-quality geometric Gaussian approximations for that family.