Geometrical Insights for Implicit Generative Modeling

TL;DR

This paper explores the geometric properties of various distances used in implicit generative models, revealing differences and providing convergence guarantees for the Wasserstein distance even with nonconvex generators.

Contribution

It offers new geometric insights into implicit generative models and establishes approximate global convergence guarantees for the 1-Wasserstein distance.

Findings

Differences in geometries induced by Wasserstein, Energy, and MMD distances.

Approximate global convergence guarantees for 1-Wasserstein distance.

Insights applicable even with nonconvex generator parametrizations.

Abstract

Learning algorithms for implicit generative models can optimize a variety of criteria that measure how the data distribution differs from the implicit model distribution, including the Wasserstein distance, the Energy distance, and the Maximum Mean Discrepancy criterion. A careful look at the geometries induced by these distances on the space of probability measures reveals interesting differences. In particular, we can establish surprising approximate global convergence guarantees for the $1$-Wasserstein distance,even when the parametric generator has a nonconvex parametrization.