A New Paradigm for Generative Adversarial Networks based on Randomized Decision Rules

TL;DR

This paper introduces a new GAN formulation based on randomized decision rules, addressing mode collapse by ensuring the discriminator converges to a fixed point and the generator reaches a Nash equilibrium, with applications in statistics.

Contribution

It proposes a novel GAN training method using empirical Bayes and MCMC, improving convergence and diversity in generated data compared to traditional GANs.

Findings

Convergence to Nash equilibrium is theoretically established.

Method improves diversity in generated samples.

Applicable to nonparametric clustering and independence testing.

Abstract

The Generative Adversarial Network (GAN) was recently introduced in the literature as a novel machine learning method for training generative models. It has many applications in statistics such as nonparametric clustering and nonparametric conditional independence tests. However, training the GAN is notoriously difficult due to the issue of mode collapse, which refers to the lack of diversity among generated data. In this paper, we identify the reasons why the GAN suffers from this issue, and to address it, we propose a new formulation for the GAN based on randomized decision rules. In the new formulation, the discriminator converges to a fixed point while the generator converges to a distribution at the Nash equilibrium. We propose to train the GAN by an empirical Bayes-like method by treating the discriminator as a hyper-parameter of the posterior distribution of the generator. Specifically, we simulate generators from its posterior distribution conditioned on the discriminator using a stochastic gradient Markov chain Monte Carlo (MCMC) algorithm, and update the discriminator using stochastic gradient descent along with simulations of the generators. We establish convergence of the proposed method to the Nash equilibrium. Apart from image generation, we apply the proposed method to nonparametric clustering and nonparametric conditional independence tests. A portion of the numerical results is presented in the supplementary material.