Regularized Finite Dimensional Kernel Sobolev Discrepancy

TL;DR

This paper clarifies that the Sobolev Discrepancy used in GANs is a weighted negative Sobolev norm related to optimal transport, and shows how to approximate it from finite samples using finite-dimensional kernels.

Contribution

It establishes the equivalence of Sobolev Discrepancy to a known Sobolev norm and provides a finite sample approximation method with error analysis.

Findings

Sobolev Discrepancy equals a weighted negative Sobolev norm.

Finite sample approximation is feasible with finite-dimensional kernels.

Error bounds depend on kernel approximation and sample size.

Abstract

We show in this note that the Sobolev Discrepancy introduced in Mroueh et al in the context of generative adversarial networks, is actually the weighted negative Sobolev norm $||.||_{\dot{H}^{-1}(\nu_q)}$, that is known to linearize the Wasserstein $W_2$ distance and plays a fundamental role in the dynamic formulation of optimal transport of Benamou and Brenier. Given a Kernel with finite dimensional feature map we show that the Sobolev discrepancy can be approximated from finite samples. Assuming this discrepancy is finite, the error depends on the approximation error in the function space induced by the finite dimensional feature space kernel and on a statistical error due to the finite sample approximation.