A Convenient Infinite Dimensional Framework for Generative Adversarial Learning

TL;DR

This paper introduces an infinite dimensional theoretical framework for GANs, establishing conditions under which the generator and discriminator converge to the true data distribution with quantifiable rates.

Contribution

It develops a novel infinite dimensional analysis for GANs, proving the optimality of the Rosenblatt transformation as a generator and convergence of Jensen-Shannon divergence.

Findings

Optimal generator via Rosenblatt transformation

Convergence of Jensen-Shannon divergence to zero

Rates of convergence under regularity assumptions

Abstract

In recent years, generative adversarial networks (GANs) have demonstrated impressive experimental results while there are only a few works that foster statistical learning theory for GANs. In this work, we propose an infinite dimensional theoretical framework for generative adversarial learning. We assume that the probability density functions of the underlying measure are uniformly bounded, $k$-times $\alpha$-H\"{o}lder differentiable ($C^{k,\alpha}$) and uniformly bounded away from zero. Under these assumptions, we show that the Rosenblatt transformation induces an optimal generator, which is realizable in the hypothesis space of $C^{k,\alpha}$-generators. With a consistent definition of the hypothesis space of discriminators, we further show that the Jensen-Shannon divergence between the distribution induced by the generator from the adversarial learning procedure and the data generating distribution converges to zero. Under certain regularity assumptions on the density of the data generating process, we also provide rates of convergence based on chaining and concentration.