Semi-Supervised Learning with IPM-based GANs: an Empirical Study

TL;DR

This paper empirically evaluates IPM-based GANs like Wasserstein, Fisher, and Sobolev GANs for semi-supervised learning, highlighting key design choices that improve performance and stability.

Contribution

It provides practical insights into the critic design for IPM-GANs in semi-supervised learning, emphasizing the importance of specific architectural choices.

Findings

K+1 formulation improves SSL performance

Avoiding batch normalization in critic enhances stability

Omitting gradient penalty on classification layer benefits results

Abstract

We present an empirical investigation of a recent class of Generative Adversarial Networks (GANs) using Integral Probability Metrics (IPM) and their performance for semi-supervised learning. IPM-based GANs like Wasserstein GAN, Fisher GAN and Sobolev GAN have desirable properties in terms of theoretical understanding, training stability, and a meaningful loss. In this work we investigate how the design of the critic (or discriminator) influences the performance in semi-supervised learning. We distill three key take-aways which are important for good SSL performance: (1) the K+1 formulation, (2) avoiding batch normalization in the critic and (3) avoiding gradient penalty constraints on the classification layer.