Rates of convergence for nonparametric estimation of singular distributions using generative adversarial networks

TL;DR

This paper analyzes the convergence rates of GANs for nonparametric distribution estimation under structural assumptions, showing they outperform VAEs and establishing bounds for optimal rates.

Contribution

It introduces convergence rate results for GANs in singular distribution estimation, surpassing VAE rates, and provides minimax lower bounds under structural assumptions.

Findings

GANs achieve faster convergence rates than VAEs under the same structural assumptions.

The paper establishes upper bounds for GAN convergence rates.

A lower bound for the minimax optimal rate is derived, conjectured to be sharp.

Abstract

It is common in nonparametric estimation problems to impose a certain low-dimensional structure on the unknown parameter to avoid the curse of dimensionality. This paper considers a nonparametric distribution estimation problem with a structural assumption under which the target distribution is allowed to be singular with respect to the Lebesgue measure. In particular, we investigate the use of generative adversarial networks (GANs) for estimating the unknown distribution and obtain a convergence rate with respect to the $L^1$-Wasserstein metric. The convergence rate depends only on the underlying structure and noise level. More interestingly, under the same structural assumption, the convergence rate of GAN is strictly faster than the known rate of VAE in the literature. We also obtain a lower bound for the minimax optimal rate, which is conjectured to be sharp at least in some special cases. Although our upper and lower bounds for the minimax optimal rate do not match, the difference is not significant.