How Does GAN-based Semi-supervised Learning Work?

TL;DR

This paper provides a theoretical analysis of GAN-based semi-supervised learning, showing how the discriminator's optimization relates to supervised learning and how unlabeled data can improve generalization.

Contribution

It offers a theoretical framework explaining how GAN-SSL enhances discriminator performance using unlabeled data, highlighting the roles of generator and discriminator optimization.

Findings

Optimal discriminator matches true data distribution under ideal conditions

Unlabeled data helps the discriminator generalize better across data manifold

GAN-SSL can improve discriminator performance without perfect generator

Abstract

Generative adversarial networks (GANs) have been widely used and have achieved competitive results in semi-supervised learning. This paper theoretically analyzes how GAN-based semi-supervised learning (GAN-SSL) works. We first prove that, given a fixed generator, optimizing the discriminator of GAN-SSL is equivalent to optimizing that of supervised learning. Thus, the optimal discriminator in GAN-SSL is expected to be perfect on labeled data. Then, if the perfect discriminator can further cause the optimization objective to reach its theoretical maximum, the optimal generator will match the true data distribution. Since it is impossible to reach the theoretical maximum in practice, one cannot expect to obtain a perfect generator for generating data, which is apparently different from the objective of GANs. Furthermore, if the labeled data can traverse all connected subdomains of the data manifold, which is reasonable in semi-supervised classification, we additionally expect the optimal discriminator in GAN-SSL to also be perfect on unlabeled data. In conclusion, the minimax optimization in GAN-SSL will theoretically output a perfect discriminator on both labeled and unlabeled data by unexpectedly learning an imperfect generator, i.e., GAN-SSL can effectively improve the generalization ability of the discriminator by leveraging unlabeled information.