Relaxed Wasserstein with Applications to GANs

TL;DR

This paper introduces Relaxed Wasserstein distances using Bregman costs to improve the flexibility and training efficiency of GANs, demonstrating superior performance on real image datasets.

Contribution

It proposes a new class of RW distances that generalize Wasserstein-1, enabling more flexible data fitting and efficient training in GANs.

Findings

RWGANs outperform WGANs on real image datasets

Using KL cost in RWGANs yields better results than gradient penalty methods

RW distances maintain statistical properties while being computationally tractable

Abstract

Wasserstein Generative Adversarial Networks (WGANs) provide a versatile class of models, which have attracted great attention in various applications. However, this framework has two main drawbacks: (i) Wasserstein-1 (or Earth-Mover) distance is restrictive such that WGANs cannot always fit data geometry well; (ii) It is difficult to achieve fast training of WGANs. In this paper, we propose a new class of \textit{Relaxed Wasserstein} (RW) distances by generalizing Wasserstein-1 distance with Bregman cost functions. We show that RW distances achieve nice statistical properties while not sacrificing the computational tractability. Combined with the GANs framework, we develop Relaxed WGANs (RWGANs) which are not only statistically flexible but can be approximated efficiently using heuristic approaches. Experiments on real images demonstrate that the RWGAN with Kullback-Leibler (KL) cost function outperforms other competing approaches, e.g., WGANs, even with gradient penalty.