On Characterizing GAN Convergence Through Proximal Duality Gap

TL;DR

This paper introduces the proximal duality gap as a new measure to monitor GAN training convergence, extending previous concepts to broader scenarios where Nash equilibria may not exist, supported by theoretical analysis and experiments.

Contribution

It extends the duality gap concept to proximal duality gap for better convergence monitoring of GANs beyond Nash equilibrium assumptions.

Findings

Proximal duality gap can monitor GAN convergence more broadly.

Theoretical link between proximal duality gap and data distribution divergence.

Experimental validation shows its effectiveness in training GANs.

Abstract

Despite the accomplishments of Generative Adversarial Networks (GANs) in modeling data distributions, training them remains a challenging task. A contributing factor to this difficulty is the non-intuitive nature of the GAN loss curves, which necessitates a subjective evaluation of the generated output to infer training progress. Recently, motivated by game theory, duality gap has been proposed as a domain agnostic measure to monitor GAN training. However, it is restricted to the setting when the GAN converges to a Nash equilibrium. But GANs need not always converge to a Nash equilibrium to model the data distribution. In this work, we extend the notion of duality gap to proximal duality gap that is applicable to the general context of training GANs where Nash equilibria may not exist. We show theoretically that the proximal duality gap is capable of monitoring the convergence of GANs to a wider spectrum of equilibria that subsumes Nash equilibria. We also theoretically establish the relationship between the proximal duality gap and the divergence between the real and generated data distributions for different GAN formulations. Our results provide new insights into the nature of GAN convergence. Finally, we validate experimentally the usefulness of proximal duality gap for monitoring and influencing GAN training.