A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training

TL;DR

This paper introduces a unified probabilistic framework called Contrastive Energy-based Models (CEM) that explains the generative ability of adversarial training and extends to unsupervised learning, improving sample quality.

Contribution

It provides the first probabilistic characterization of adversarial training and develops a unified framework that enhances both supervised and unsupervised generative modeling.

Findings

Improved sample quality in supervised and unsupervised settings.

Unsupervised adversarial sampling achieves high Inception score of 9.61 on CIFAR-10.

Framework outperforms previous energy-based models.

Abstract

Adversarial Training (AT) is known as an effective approach to enhance the robustness of deep neural networks. Recently researchers notice that robust models with AT have good generative ability and can synthesize realistic images, while the reason behind it is yet under-explored. In this paper, we demystify this phenomenon by developing a unified probabilistic framework, called Contrastive Energy-based Models (CEM). On the one hand, we provide the first probabilistic characterization of AT through a unified understanding of robustness and generative ability. On the other hand, our unified framework can be extended to the unsupervised scenario, which interprets unsupervised contrastive learning as an important sampling of CEM. Based on these, we propose a principled method to develop adversarial learning and sampling methods. Experiments show that the sampling methods derived from our framework improve the sample quality in both supervised and unsupervised learning. Notably, our unsupervised adversarial sampling method achieves an Inception score of 9.61 on CIFAR-10, which is superior to previous energy-based models and comparable to state-of-the-art generative models.