A Probe Towards Understanding GAN and VAE Models

TL;DR

This paper compares GAN and VAE models up to 2017, analyzing their differences in data distribution approximation, and introduces a new model based on observed behavioral hypotheses, tested on MNIST and CelebA datasets.

Contribution

It provides a comparative analysis of GAN and VAE models, offers a hypothesis explaining their behavioral differences, and proposes a new model validated on benchmark datasets.

Findings

GANs and VAEs exhibit different behaviors in fidelity and mode collapse.

A hypothesis explains the behavioral differences between GANs and VAEs.

The proposed model shows promising results on MNIST and CelebA datasets.

Abstract

This project report compares some known GAN and VAE models proposed prior to 2017. There has been significant progress after we finished this report. We upload this report as an introduction to generative models and provide some personal interpretations supported by empirical evidence. Both generative adversarial network models and variational autoencoders have been widely used to approximate probability distributions of data sets. Although they both use parametrized distributions to approximate the underlying data distribution, whose exact inference is intractable, their behaviors are very different. We summarize our experiment results that compare these two categories of models in terms of fidelity and mode collapse. We provide a hypothesis to explain their different behaviors and propose a new model based on this hypothesis. We further tested our proposed model on MNIST dataset and CelebA dataset.