Non-Asymptotic Error Bounds for Bidirectional GANs

TL;DR

This paper provides the first theoretical error bounds for bidirectional GANs, applicable without assumptions of equal dimensions or bounded support, enhancing understanding of their learning guarantees.

Contribution

It introduces nearly sharp non-asymptotic bounds for BiGANs, including for Wasserstein variants, without common restrictive assumptions, and develops new techniques for error analysis.

Findings

First theoretical guarantees for BiGANs.

Bounds applicable to unbounded and different-dimensional distributions.

Method extends to Wasserstein BiGANs with bounded support.

Abstract

We derive nearly sharp bounds for the bidirectional GAN (BiGAN) estimation error under the Dudley distance between the latent joint distribution and the data joint distribution with appropriately specified architecture of the neural networks used in the model. To the best of our knowledge, this is the first theoretical guarantee for the bidirectional GAN learning approach. An appealing feature of our results is that they do not assume the reference and the data distributions to have the same dimensions or these distributions to have bounded support. These assumptions are commonly assumed in the existing convergence analysis of the unidirectional GANs but may not be satisfied in practice. Our results are also applicable to the Wasserstein bidirectional GAN if the target distribution is assumed to have a bounded support. To prove these results, we construct neural network functions that push forward an empirical distribution to another arbitrary empirical distribution on a possibly different-dimensional space. We also develop a novel decomposition of the integral probability metric for the error analysis of bidirectional GANs. These basic theoretical results are of independent interest and can be applied to other related learning problems.