$\alpha$-GAN: Convergence and Estimation Guarantees

TL;DR

This paper establishes theoretical convergence guarantees for $oldsymbol{ extalpha}$-GANs, a family of generative adversarial networks interpolating various divergence measures, and demonstrates practical benefits of tuning the $oldsymbol{ extalpha}$ hyperparameter.

Contribution

It provides a theoretical link between min-max GAN optimization and $f$-divergences, introduces $oldsymbol{ extalpha}$-GANs interpolating multiple divergences, and offers empirical insights on hyperparameter tuning.

Findings

$oldsymbol{ extalpha}$-GANs converge for all $oldsymbol{ extalpha}$ values.

Estimation bounds vary with $oldsymbol{ extalpha}$ under finite samples.

Tuning $oldsymbol{ extalpha}$ improves practical performance.

Abstract

We prove a two-way correspondence between the min-max optimization of general CPE loss function GANs and the minimization of associated $f$-divergences. We then focus on $\alpha$-GAN, defined via the $\alpha$-loss, which interpolates several GANs (Hellinger, vanilla, Total Variation) and corresponds to the minimization of the Arimoto divergence. We show that the Arimoto divergences induced by $\alpha$-GAN equivalently converge, for all $\alpha\in \mathbb{R}_{>0}\cup\{\infty\}$. However, under restricted learning models and finite samples, we provide estimation bounds which indicate diverse GAN behavior as a function of $\alpha$. Finally, we present empirical results on a toy dataset that highlight the practical utility of tuning the $\alpha$ hyperparameter.