Out-of-Sample Testing for GANs

TL;DR

This paper introduces EvalGAN, a novel evaluation method for GANs that measures reconstruction quality and likelihood directly from a test set, applicable across different GAN algorithms and datasets.

Contribution

EvalGAN provides a dataset-agnostic, direct evaluation approach for GANs without auxiliary networks, improving assessment of generative quality.

Findings

EvalGAN successfully evaluates three state-of-the-art GANs.

It measures reconstruction quality directly in the original sample space.

It computes likelihoods for reconstructed samples.

Abstract

We propose a new method to evaluate GANs, namely EvalGAN. EvalGAN relies on a test set to directly measure the reconstruction quality in the original sample space (no auxiliary networks are necessary), and it also computes the (log)likelihood for the reconstructed samples in the test set. Further, EvalGAN is agnostic to the GAN algorithm and the dataset. We decided to test it on three state-of-the-art GANs over the well-known CIFAR-10 and CelebA datasets.