Variational inference via radial transport

TL;DR

This paper introduces radVI, a novel variational inference method focusing on optimizing radial profiles of distributions, with theoretical guarantees and compatibility with existing VI schemes.

Contribution

The paper proposes radVI, a new radial transport-based algorithm for variational inference with convergence guarantees and broad applicability.

Findings

radVI effectively improves coverage in VI by optimizing radial profiles.

Theoretical convergence guarantees are established for radVI.

radVI can be integrated with Gaussian VI and Laplace approximation.

Abstract

In variational inference (VI), the practitioner approximates a high-dimensional distribution $\pi$ with a simple surrogate one, often a (product) Gaussian distribution. However, in many cases of practical interest, Gaussian distributions might not capture the correct radial profile of $\pi$, resulting in poor coverage. In this work, we approach the VI problem from the perspective of optimizing over these radial profiles. Our algorithm radVI is a cheap, effective add-on to many existing VI schemes, such as Gaussian (mean-field) VI and Laplace approximation. We provide theoretical convergence guarantees for our algorithm, owing to recent developments in optimization over the Wasserstein space--the space of probability distributions endowed with the Wasserstein distance--and new regularity properties of radial transport maps in the style of Caffarelli (2000).