Convergence Problems with Generative Adversarial Networks (GANs)

TL;DR

This paper reviews the convergence issues in GAN training, exploring theoretical challenges and recent advances from topological and game-theoretic perspectives to improve understanding and techniques.

Contribution

It provides a comprehensive theoretical account of GAN convergence problems, highlighting recent insights from topology and game theory that enhance training stability.

Findings

Identification of key convergence problems in GAN training

Analysis of topological and game-theoretic approaches to GANs

Discussion of recent techniques improving GAN training stability

Abstract

Generative adversarial networks (GANs) are a novel approach to generative modelling, a task whose goal it is to learn a distribution of real data points. They have often proved difficult to train: GANs are unlike many techniques in machine learning, in that they are best described as a two-player game between a discriminator and generator. This has yielded both unreliability in the training process, and a general lack of understanding as to how GANs converge, and if so, to what. The purpose of this dissertation is to provide an account of the theory of GANs suitable for the mathematician, highlighting both positive and negative results. This involves identifying the problems when training GANs, and how topological and game-theoretic perspectives of GANs have contributed to our understanding and improved our techniques in recent years.