Diagnosing and Enhancing VAE Models

TL;DR

This paper provides a detailed analysis of variational autoencoders, challenges common assumptions about their limitations, and introduces a simple enhancement that improves sample quality and stability, rivaling GANs without extra tuning.

Contribution

It offers a rigorous analysis of VAE objectives and proposes a hyperparameter-free enhancement that improves sample quality and stability.

Findings

Enhanced VAE produces crisp samples

Achieves stable FID scores comparable to GANs

No additional hyperparameters required

Abstract

Although variational autoencoders (VAEs) represent a widely influential deep generative model, many aspects of the underlying energy function remain poorly understood. In particular, it is commonly believed that Gaussian encoder/decoder assumptions reduce the effectiveness of VAEs in generating realistic samples. In this regard, we rigorously analyze the VAE objective, differentiating situations where this belief is and is not actually true. We then leverage the corresponding insights to develop a simple VAE enhancement that requires no additional hyperparameters or sensitive tuning. Quantitatively, this proposal produces crisp samples and stable FID scores that are actually competitive with a variety of GAN models, all while retaining desirable attributes of the original VAE architecture. A shorter version of this work will appear in the ICLR 2019 conference proceedings (Dai and Wipf, 2019). The code for our model is available at https://github.com/daib13/ TwoStageVAE.