WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points

TL;DR

This paper proves that GANs with an infinitely wide generator and a finite-width discriminator have no spurious stationary points, providing insights into their training dynamics and stability.

Contribution

It establishes the absence of spurious stationary points in GANs with an infinitely wide generator, advancing theoretical understanding of GAN training.

Findings

Infinite-width generator GANs have no spurious stationary points.

Finite but wide generator GANs lack spurious stationary points within large regions.

Results improve understanding of GAN convergence properties.

Abstract

Generative adversarial networks (GAN) are a widely used class of deep generative models, but their minimax training dynamics are not understood very well. In this work, we show that GANs with a 2-layer infinite-width generator and a 2-layer finite-width discriminator trained with stochastic gradient ascent-descent have no spurious stationary points. We then show that when the width of the generator is finite but wide, there are no spurious stationary points within a ball whose radius becomes arbitrarily large (to cover the entire parameter space) as the width goes to infinity.