Nonparametric Density Estimation under Adversarial Losses

TL;DR

This paper analyzes the minimax convergence rates for nonparametric density estimation under various adversarial losses, including MMD and Wasserstein, with implications for GAN training and implicit generative models.

Contribution

It introduces a general framework linking loss choice, density smoothness, and minimax rates, extending understanding of adversarial losses in density estimation.

Findings

Characterizes minimax rates for various adversarial losses.

Connects loss functions to GAN training dynamics.

Provides insights into learning implicit generative models.

Abstract

We study minimax convergence rates of nonparametric density estimation under a large class of loss functions called "adversarial losses", which, besides classical $\mathcal{L}^p$ losses, includes maximum mean discrepancy (MMD), Wasserstein distance, and total variation distance. These losses are closely related to the losses encoded by discriminator networks in generative adversarial networks (GANs). In a general framework, we study how the choice of loss and the assumed smoothness of the underlying density together determine the minimax rate. We also discuss implications for training GANs based on deep ReLU networks, and more general connections to learning implicit generative models in a minimax statistical sense.