Good Semi-supervised Learning that Requires a Bad GAN

TL;DR

This paper reveals that effective semi-supervised learning with GANs requires a deliberately poor generator, and introduces a new method that outperforms existing models on benchmark datasets.

Contribution

It provides a theoretical explanation for why a bad generator benefits semi-supervised learning and proposes a novel formulation that achieves state-of-the-art results.

Findings

Theoretically shows that good semi-supervised learning needs a bad generator.

Empirically improves over feature matching GANs.

Achieves state-of-the-art results on multiple benchmarks.

Abstract

Semi-supervised learning methods based on generative adversarial networks (GANs) obtained strong empirical results, but it is not clear 1) how the discriminator benefits from joint training with a generator, and 2) why good semi-supervised classification performance and a good generator cannot be obtained at the same time. Theoretically, we show that given the discriminator objective, good semisupervised learning indeed requires a bad generator, and propose the definition of a preferred generator. Empirically, we derive a novel formulation based on our analysis that substantially improves over feature matching GANs, obtaining state-of-the-art results on multiple benchmark datasets.