Generative Modeling by Minimizing the Wasserstein-2 Loss

TL;DR

This paper introduces a novel generative modeling approach that minimizes the Wasserstein-2 loss using a distribution-dependent ODE, demonstrating exponential convergence and improved performance over Wasserstein GANs.

Contribution

It develops a new gradient flow-based algorithm for Wasserstein-2 minimization, with theoretical convergence guarantees and practical improvements in experiments.

Findings

The proposed method converges exponentially to the true data distribution.

The Euler scheme effectively approximates the gradient flow for $W_2$ loss.

Our algorithm outperforms Wasserstein GANs in both low- and high-dimensional settings.

Abstract

This paper develops a generative model by minimizing the second-order Wasserstein loss (the $W_2$ loss) through a distribution-dependent ordinary differential equation (ODE), whose dynamics involves the Kantorovich potential associated with the true data distribution and a current estimate of it. A main result shows that the time-marginal laws of the ODE form a gradient flow for the $W_2$ loss, which converges exponentially to the true data distribution. An Euler scheme for the ODE is proposed and it is shown to recover the gradient flow for the $W_2$ loss in the limit. An algorithm is designed by following the scheme and applying persistent training, which naturally fits our gradient-flow approach. In both low- and high-dimensional experiments, our algorithm outperforms Wasserstein generative adversarial networks by increasing the level of persistent training appropriately.