Is Supervised Learning With Adversarial Features Provably Better Than Sole Supervision?

TL;DR

This paper provides a theoretical analysis demonstrating that supervised learning augmented with adversarial features can outperform traditional supervised learning, especially by mitigating vanishing gradient issues near optimal solutions.

Contribution

The paper proves that supervised learning with adversarial features can be provably better than sole supervision under certain mild assumptions.

Findings

Supervised learning without adversarial features suffers from vanishing gradients near optimality.

Adversarial learning with supervised signals mitigates the vanishing gradient problem.

Supervised learning with adversarial features can outperform sole supervision in empirical risk and convergence rate.

Abstract

Generative Adversarial Networks (GAN) have shown promising results on a wide variety of complex tasks. Recent experiments show adversarial training provides useful gradients to the generator that helps attain better performance. In this paper, we intend to theoretically analyze whether supervised learning with adversarial features can outperform sole supervision, or not. First, we show that supervised learning without adversarial features suffer from vanishing gradient issue in near optimal region. Second, we analyze how adversarial learning augmented with supervised signal mitigates this vanishing gradient issue. Finally, we prove our main result that shows supervised learning with adversarial features can be better than sole supervision (under some mild assumptions). We support our main result on two fronts (i) expected empirical risk and (ii) rate of convergence.