Wasserstein Auto-Encoders

TL;DR

The paper introduces Wasserstein Auto-Encoders (WAE), a new generative model that minimizes Wasserstein distance with a novel regularizer, producing higher quality samples than VAEs and sharing many of their desirable properties.

Contribution

It presents WAE, a novel algorithm that generalizes adversarial auto-encoders by using Wasserstein distance, leading to improved sample quality and stable training.

Findings

WAE achieves better sample quality than VAEs as measured by FID score.

WAE maintains stable training and a meaningful latent space.

WAE generalizes adversarial auto-encoders with a different regularizer.

Abstract

We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Auto-Encoder (VAE). This regularizer encourages the encoded training distribution to match the prior. We compare our algorithm with several other techniques and show that it is a generalization of adversarial auto-encoders (AAE). Our experiments show that WAE shares many of the properties of VAEs (stable training, encoder-decoder architecture, nice latent manifold structure) while generating samples of better quality, as measured by the FID score.