A Unifying Generator Loss Function for Generative Adversarial Networks

TL;DR

This paper introduces a unifying $ ext{L}_ ext{alpha}$-GAN framework with a parameterized generator loss that generalizes many existing GAN variants, supported by theoretical analysis and experiments on standard datasets.

Contribution

It proposes a new $ ext{L}_ ext{alpha}$-parameterized generator loss function that unifies multiple GAN variants under a common theoretical framework.

Findings

The $ ext{L}_ ext{alpha}$-GAN minimizes a Jensen-$f_ ext{alpha}$-divergence.

The framework recovers VanillaGAN, LSGAN, L$k$GAN, and $( ext{alpha}_D, ext{alpha}_G)$-GAN as special cases.

Experimental results demonstrate the effectiveness of the proposed system on MNIST, CIFAR-10, and Stacked MNIST datasets.

Abstract

A unifying $\alpha$-parametrized generator loss function is introduced for a dual-objective generative adversarial network (GAN), which uses a canonical (or classical) discriminator loss function such as the one in the original GAN (VanillaGAN) system. The generator loss function is based on a symmetric class probability estimation type function, $\mathcal{L}_\alpha$, and the resulting GAN system is termed $\mathcal{L}_\alpha$-GAN. Under an optimal discriminator, it is shown that the generator's optimization problem consists of minimizing a Jensen-$f_\alpha$-divergence, a natural generalization of the Jensen-Shannon divergence, where $f_\alpha$ is a convex function expressed in terms of the loss function $\mathcal{L}_\alpha$. It is also demonstrated that this $\mathcal{L}_\alpha$-GAN problem recovers as special cases a number of GAN problems in the literature, including VanillaGAN, Least Squares GAN (LSGAN), Least $k$th order GAN (L$k$GAN) and the recently introduced $(\alpha_D,\alpha_G)$-GAN with $\alpha_D=1$. Finally, experimental results are conducted on three datasets, MNIST, CIFAR-10, and Stacked MNIST to illustrate the performance of various examples of the $\mathcal{L}_\alpha$-GAN system.