Approximating Probability Distributions by using Wasserstein Generative Adversarial Networks

TL;DR

This paper analyzes Wasserstein GANs with GroupSort discriminators, providing theoretical bounds on their approximation error and showing how discriminator capacity affects performance, supported by experiments on Swiss roll and MNIST datasets.

Contribution

It establishes a theoretical generalization bound for WGANs and highlights the importance of discriminator capacity relative to generator capacity.

Findings

Discriminator capacity must be sufficiently high for effective approximation.

Overly deep and wide generators can perform worse if discriminators are weak.

Experimental results confirm the theoretical bounds.

Abstract

Studied here are Wasserstein generative adversarial networks (WGANs) with GroupSort neural networks as their discriminators. It is shown that the error bound of the approximation for the target distribution depends on the width and depth (capacity) of the generators and discriminators and the number of samples in training. A quantified generalization bound is established for the Wasserstein distance between the generated and target distributions. According to the theoretical results, WGANs have a higher requirement for the capacity of discriminators than that of generators, which is consistent with some existing results. More importantly, the results with overly deep and wide (high-capacity) generators may be worse than those with low-capacity generators if discriminators are insufficiently strong. Numerical results obtained using Swiss roll and MNIST datasets confirm the theoretical results.