Failure Modes of Variational Autoencoders and Their Effects on Downstream Tasks

TL;DR

This paper characterizes specific failure modes of Variational Autoencoders (VAEs) and analyzes how these issues affect their performance on downstream tasks like representation learning, robustness, and semi-supervised learning.

Contribution

It provides concrete conditions under which VAE training fails and links these failures to their impact on various downstream applications.

Findings

Identifies conditions leading to VAE training pathologies

Connects failure modes to downstream task performance

Provides insights for improving VAE robustness

Abstract

Variational Auto-encoders (VAEs) are deep generative latent variable models that are widely used for a number of downstream tasks. While it has been demonstrated that VAE training can suffer from a number of pathologies, existing literature lacks characterizations of exactly when these pathologies occur and how they impact downstream task performance. In this paper, we concretely characterize conditions under which VAE training exhibits pathologies and connect these failure modes to undesirable effects on specific downstream tasks, such as learning compressed and disentangled representations, adversarial robustness, and semi-supervised learning.