Lipschitz Constrained GANs via Boundedness and Continuity

TL;DR

This paper introduces BC conditions to enforce Lipschitz constraints in GAN discriminators, providing a theoretically sound and computationally efficient method that outperforms existing techniques like gradient penalty and spectral normalization.

Contribution

It proposes BC conditions for Lipschitz enforcement in GANs, with theoretical proof and a practical CNN-based implementation that improves performance and reduces complexity.

Findings

BC-GANs outperform recent techniques in performance.

BC-GANs have lower computational complexity.

Theoretical proof that BC conditions ensure Lipschitz constraints.

Abstract

One of the challenges in the study of Generative Adversarial Networks (GANs) is the difficulty of its performance control. Lipschitz constraint is essential in guaranteeing training stability for GANs. Although heuristic methods such as weight clipping, gradient penalty and spectral normalization have been proposed to enforce Lipschitz constraint, it is still difficult to achieve a solution that is both practically effective and theoretically provably satisfying a Lipschitz constraint. In this paper, we introduce the boundedness and continuity ($BC$) conditions to enforce the Lipschitz constraint on the discriminator functions of GANs. We prove theoretically that GANs with discriminators meeting the BC conditions satisfy the Lipschitz constraint. We present a practically very effective implementation of a GAN based on a convolutional neural network (CNN) by forcing the CNN to satisfy the $BC$ conditions (BC-GAN). We show that as compared to recent techniques including gradient penalty and spectral normalization, BC-GANs not only have better performances but also lower computational complexity.