Realizing GANs via a Tunable Loss Function

TL;DR

This paper introduces $oldsymbol{ extalpha}$-GAN, a tunable generative adversarial network that interpolates between different GAN variants using a parameterized loss function, aiming to improve training stability and diversity.

Contribution

The paper proposes $oldsymbol{ extalpha}$-GAN, a novel GAN framework with a tunable loss function that unifies various GAN types and explores its theoretical properties and convergence behavior.

Findings

$oldsymbol{ extalpha}$-GAN interpolates between $f$-GANs and IPM-based GANs.

The $oldsymbol{ extalpha}$-loss captures multiple canonical losses.

Theoretical analysis links $oldsymbol{ extalpha}$-GAN to Arimoto divergence.

Abstract

We introduce a tunable GAN, called $\alpha$-GAN, parameterized by $\alpha \in (0,\infty]$, which interpolates between various $f$-GANs and Integral Probability Metric based GANs (under constrained discriminator set). We construct $\alpha$-GAN using a supervised loss function, namely, $\alpha$-loss, which is a tunable loss function capturing several canonical losses. We show that $\alpha$-GAN is intimately related to the Arimoto divergence, which was first proposed by \"{O}sterriecher (1996), and later studied by Liese and Vajda (2006). We also study the convergence properties of $\alpha$-GAN. We posit that the holistic understanding that $\alpha$-GAN introduces will have practical benefits of addressing both the issues of vanishing gradients and mode collapse.