Sobolev GAN

TL;DR

This paper introduces Sobolev IPM, a new metric for high-dimensional distribution comparison, and demonstrates its application in training GANs, text generation, and semi-supervised learning with competitive results.

Contribution

The paper presents Sobolev IPM, a novel high-dimensional distribution metric, and applies it to improve GAN training, text generation, and semi-supervised learning.

Findings

Sobolev IPM effectively compares high-dimensional distributions using weighted conditional CDFs.

Sobolev GAN achieves competitive semi-supervised learning results on CIFAR-10.

The approach enforces smoothness in the critic, enhancing GAN training stability.

Abstract

We propose a new Integral Probability Metric (IPM) between distributions: the Sobolev IPM. The Sobolev IPM compares the mean discrepancy of two distributions for functions (critic) restricted to a Sobolev ball defined with respect to a dominant measure $\mu$. We show that the Sobolev IPM compares two distributions in high dimensions based on weighted conditional Cumulative Distribution Functions (CDF) of each coordinate on a leave one out basis. The Dominant measure $\mu$ plays a crucial role as it defines the support on which conditional CDFs are compared. Sobolev IPM can be seen as an extension of the one dimensional Von-Mises Cram\'er statistics to high dimensional distributions. We show how Sobolev IPM can be used to train Generative Adversarial Networks (GANs). We then exploit the intrinsic conditioning implied by Sobolev IPM in text generation. Finally we show that a variant of Sobolev GAN achieves competitive results in semi-supervised learning on CIFAR-10, thanks to the smoothness enforced on the critic by Sobolev GAN which relates to Laplacian regularization.