Flooding Regularization for Stable Training of Generative Adversarial Networks

TL;DR

This paper introduces flooding regularization for GANs, which stabilizes training by preventing the discriminator's loss from becoming too low, supported by theoretical analysis and experimental validation.

Contribution

It proposes a novel flooding regularization method for GANs that directly stabilizes training by controlling the discriminator's loss, with theoretical and empirical support.

Findings

Flooding stabilizes GAN training.

The method can be combined with other stabilization techniques.

Stable training occurs even with high flood levels.

Abstract

Generative Adversarial Networks (GANs) have shown remarkable performance in image generation. However, GAN training suffers from the problem of instability. One of the main approaches to address this problem is to modify the loss function, often using regularization terms in addition to changing the type of adversarial losses. This paper focuses on directly regularizing the adversarial loss function. We propose a method that applies flooding, an overfitting suppression method in supervised learning, to GANs to directly prevent the discriminator's loss from becoming excessively low. Flooding requires tuning the flood level, but when applied to GANs, we propose that the appropriate range of flood level settings is determined by the adversarial loss function, supported by theoretical analysis of GANs using the binary cross entropy loss. We experimentally verify that flooding stabilizes GAN training and can be combined with other stabilization techniques. We also show that by restricting the discriminator's loss to be no less than the flood level, the training proceeds stably even when the flood level is somewhat high.