Consistency Regularization for Generative Adversarial Networks

TL;DR

This paper introduces a simple consistency regularization technique for GAN training that stabilizes learning, improves image quality, and enhances state-of-the-art FID scores across multiple datasets and architectures.

Contribution

It proposes a novel regularization method based on consistency regularization that effectively stabilizes GAN training and improves generation quality.

Findings

Achieves best FID scores on CIFAR-10 and CelebA datasets.

Improves state-of-the-art FID scores for conditional image generation.

Works effectively with spectral normalization and various GAN architectures.

Abstract

Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort. Several regularization techniques for stabilizing training have been proposed, but they introduce non-trivial computational overheads and interact poorly with existing techniques like spectral normalization. In this work, we propose a simple, effective training stabilizer based on the notion of consistency regularization---a popular technique in the semi-supervised learning literature. In particular, we augment data passing into the GAN discriminator and penalize the sensitivity of the discriminator to these augmentations. We conduct a series of experiments to demonstrate that consistency regularization works effectively with spectral normalization and various GAN architectures, loss functions and optimizer settings. Our method achieves the best FID scores for unconditional image generation compared to other regularization methods on CIFAR-10 and CelebA. Moreover, Our consistency regularized GAN (CR-GAN) improves state-of-the-art FID scores for conditional generation from 14.73 to 11.48 on CIFAR-10 and from 8.73 to 6.66 on ImageNet-2012.