A New Perspective on Stabilizing GANs training: Direct Adversarial Training

TL;DR

This paper introduces Direct Adversarial Training (DAT), a novel method to stabilize GAN training by addressing the adversarial nature of generated images, demonstrating significant improvements across multiple datasets and architectures.

Contribution

The paper presents a new perspective on GAN stabilization by identifying the adversarial role of generated images and proposes DAT to adaptively minimize the discriminator's Lipschitz constant.

Findings

DAT improves FID scores significantly on CIFAR-100, STL-10, and LSUN-Bedroom datasets.

The method is effective across various loss functions, architectures, and hyper-parameters.

Experimental results confirm the stability and performance enhancement of GAN training with DAT.

Abstract

Generative Adversarial Networks (GANs) are the most popular image generation models that have achieved remarkable progress on various computer vision tasks. However, training instability is still one of the open problems for all GAN-based algorithms. Quite a number of methods have been proposed to stabilize the training of GANs, the focuses of which were respectively put on the loss functions, regularization and normalization technologies, training algorithms, and model architectures. Different from the above methods, in this paper, a new perspective on stabilizing GANs training is presented. It is found that sometimes the images produced by the generator act like adversarial examples of the discriminator during the training process, which may be part of the reason causing the unstable training of GANs. With this finding, we propose the Direct Adversarial Training (DAT) method to stabilize the training process of GANs. Furthermore, we prove that the DAT method is able to minimize the Lipschitz constant of the discriminator adaptively. The advanced performance of DAT is verified on multiple loss functions, network architectures, hyper-parameters, and datasets. Specifically, DAT achieves significant improvements of 11.5% FID on CIFAR-100 unconditional generation based on SSGAN, 10.5% FID on STL-10 unconditional generation based on SSGAN, and 13.2% FID on LSUN-Bedroom unconditional generation based on SSGAN. Code will be available at https://github.com/iceli1007/DAT-GAN