# $(\alpha_D,\alpha_G)$-GANs: Addressing GAN Training Instabilities via Dual Objectives

## TL;DR

This paper proposes a novel class of dual-objective GANs using $oldsymbol{ ext{α-loss}}$ functions, which improves training stability by tuning objectives for the generator and discriminator, with theoretical analysis and empirical validation.

## Contribution

It introduces $(oldsymbol{ ext{α}_D, ext{α}_G})$-GANs with tunable objectives, providing theoretical insights and bounds, and demonstrates enhanced training stability through experiments.

## Key findings

- The non-zero sum game simplifies to $f$-divergence minimization under certain conditions.
- Upper bounds on estimation error are order optimal in finite sample and capacity settings.
- Tuning $( ext{α}_D, ext{α}_G)$) alleviates training instabilities in experiments.

## Abstract

In an effort to address the training instabilities of GANs, we introduce a class of dual-objective GANs with different value functions (objectives) for the generator (G) and discriminator (D). In particular, we model each objective using $\alpha$-loss, a tunable classification loss, to obtain $(\alpha_D,\alpha_G)$-GANs, parameterized by $(\alpha_D,\alpha_G)\in (0,\infty]^2$. For sufficiently large number of samples and capacities for G and D, we show that the resulting non-zero sum game simplifies to minimizing an $f$-divergence under appropriate conditions on $(\alpha_D,\alpha_G)$. In the finite sample and capacity setting, we define estimation error to quantify the gap in the generator's performance relative to the optimal setting with infinite samples and obtain upper bounds on this error, showing it to be order optimal under certain conditions. Finally, we highlight the value of tuning $(\alpha_D,\alpha_G)$ in alleviating training instabilities for the synthetic 2D Gaussian mixture ring and the Stacked MNIST datasets.