Variance Regularizing Adversarial Learning

TL;DR

This paper proposes a new adversarial training method that uses a bi-modal Gaussian distribution to improve gradient flow and stability in generative adversarial networks, demonstrated on image datasets.

Contribution

It introduces a variance regularizing approach with a Gaussian classifier to enhance adversarial training stability and gradient quality.

Findings

The method maintains non-zero gradients even with a perfect classifier.

The classifier output distribution is smooth with overlap between real and fake modes.

Effective on standard benchmark image datasets.

Abstract

We introduce a novel approach for training adversarial models by replacing the discriminator score with a bi-modal Gaussian distribution over the real/fake indicator variables. In order to do this, we train the Gaussian classifier to match the target bi-modal distribution implicitly through meta-adversarial training. We hypothesize that this approach ensures a non-zero gradient to the generator, even in the limit of a perfect classifier. We test our method against standard benchmark image datasets as well as show the classifier output distribution is smooth and has overlap between the real and fake modes.