Wasserstein Divergence for GANs

TL;DR

This paper introduces WGAN-div, a new divergence measure for GANs that relaxes the Lipschitz constraint, leading to more stable training and superior image synthesis performance.

Contribution

The paper proposes WGAN-div, a novel Wasserstein divergence that simplifies training by removing the Lipschitz constraint, and demonstrates its effectiveness in stable training and high-quality image generation.

Findings

WGAN-div achieves more stable training than traditional WGANs.

WGAN-div outperforms state-of-the-art methods on image synthesis benchmarks.

Theoretical analysis shows WGAN-div's advantages over WGANs.

Abstract

In many domains of computer vision, generative adversarial networks (GANs) have achieved great success, among which the family of Wasserstein GANs (WGANs) is considered to be state-of-the-art due to the theoretical contributions and competitive qualitative performance. However, it is very challenging to approximate the $k$-Lipschitz constraint required by the Wasserstein-1 metric~(W-met). In this paper, we propose a novel Wasserstein divergence~(W-div), which is a relaxed version of W-met and does not require the $k$-Lipschitz constraint. As a concrete application, we introduce a Wasserstein divergence objective for GANs~(WGAN-div), which can faithfully approximate W-div through optimization. Under various settings, including progressive growing training, we demonstrate the stability of the proposed WGAN-div owing to its theoretical and practical advantages over WGANs. Also, we study the quantitative and visual performance of WGAN-div on standard image synthesis benchmarks of computer vision, showing the superior performance of WGAN-div compared to the state-of-the-art methods.