Distribution Matching in Variational Inference

TL;DR

This paper analyzes the limitations of Variational Autoencoders in learning distributions, evaluates VAE-GAN hybrids, and discusses their practical challenges and limited improvements over GANs.

Contribution

It reveals the fundamental issues in VAEs related to distribution matching and provides a comprehensive evaluation of VAE-GAN hybrids highlighting their practical limitations.

Findings

VAEs fail to learn marginal distributions effectively.

VAE-GAN hybrids are harder to scale and evaluate.

VAE-GANs do not outperform GANs in generation quality.

Abstract

With the increasingly widespread deployment of generative models, there is a mounting need for a deeper understanding of their behaviors and limitations. In this paper, we expose the limitations of Variational Autoencoders (VAEs), which consistently fail to learn marginal distributions in both latent and visible spaces. We show this to be a consequence of learning by matching conditional distributions, and the limitations of explicit model and posterior distributions. It is popular to consider Generative Adversarial Networks (GANs) as a means of overcoming these limitations, leading to hybrids of VAEs and GANs. We perform a large-scale evaluation of several VAE-GAN hybrids and analyze the implications of class probability estimation for learning distributions. While promising, we conclude that at present, VAE-GAN hybrids have limited applicability: they are harder to scale, evaluate, and use for inference compared to VAEs; and they do not improve over the generation quality of GANs.