On the estimation of the Wasserstein distance in generative models

TL;DR

This paper explores methods for estimating the Wasserstein distance in generative models, particularly GANs, discusses training challenges, and proposes extensions to improve model performance and stability.

Contribution

It systematically analyzes Wasserstein distance estimation techniques and extends existing methods to various algorithms, cost functions, and regularization schemes.

Findings

Various Wasserstein distance estimation methods are compared.

Training secrets and best practices are summarized.

Extensions improve generative model stability and quality.

Abstract

Generative Adversarial Networks (GANs) have been used to model the underlying probability distribution of sample based datasets. GANs are notoriuos for training difficulties and their dependence on arbitrary hyperparameters. One recent improvement in GAN literature is to use the Wasserstein distance as loss function leading to Wasserstein Generative Adversarial Networks (WGANs). Using this as a basis, we show various ways in which the Wasserstein distance is estimated for the task of generative modelling. Additionally, the secrets in training such models are shown and summarized at the end of this work. Where applicable, we extend current works to different algorithms, different cost functions, and different regularization schemes to improve generative models.