Stochastic moments dynamics: a flexible finite-dimensional random perturbation of Wasserstein gradient descent

TL;DR

This paper introduces a stochastic process that perturbs the moments of probability distributions using finite-dimensional diffusions, enabling better exploration in distribution optimization, with theoretical guarantees up to explosion time.

Contribution

It proposes a novel finite-dimensional stochastic process for distribution optimization, extending previous models and analyzing its well-posedness and mean-field behavior.

Findings

The process can explode in finite time under certain conditions.

Well-posedness and propagation of chaos are established up to explosion time.

The method generalizes previous Brownian noise approaches in distribution space.

Abstract

For optimizing a non-convex function in finite dimension, a method is to add Brownian noise to a gradient descent, allowing for transitions between basins of attractions of different minimizers. To adapt this for optimization over a space of probability distributions requires a suitable noise. For this purpose, we introduce here a simple stochastic process where a number of moments of the distribution are following a chosen finite-dimensional diffusion process, generalizing some previous studies where the expectation of the measure is subject to a Brownian noise. The process may explode in finite time, for instance when trying to force the variance of a distribution to behave like a Brownian motion. We show, up to the possible explosion time, well-posedness and propagation of chaos for the system of mean-field interacting particles with common noise approximating the process.