Comparison of Maximum Likelihood and GAN-based training of Real NVPs

TL;DR

This paper compares maximum likelihood and Wasserstein GAN training methods for Real NVP models, analyzing generated samples, densities, and distances, and introduces a fast learning critic for improved detection of overfitting.

Contribution

It provides a comparative analysis of MLE and GAN training for Real NVPs and proposes a novel fast learning critic to detect overfitting efficiently.

Findings

GAN-trained models produce different sample qualities than MLE-trained models

A critic trained to estimate Wasserstein distance helps detect overfitting

The fast learning critic offers a quick method for overfitting detection

Abstract

We train a generator by maximum likelihood and we also train the same generator architecture by Wasserstein GAN. We then compare the generated samples, exact log-probability densities and approximate Wasserstein distances. We show that an independent critic trained to approximate Wasserstein distance between the validation set and the generator distribution helps detect overfitting. Finally, we use ideas from the one-shot learning literature to develop a novel fast learning critic.