Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model

TL;DR

This paper introduces Sliced-Wasserstein Autoencoders (SWAE), a simple yet effective generative model that shapes latent space distributions using sliced-Wasserstein distance without adversarial training.

Contribution

The paper proposes SWAE, a novel autoencoder-based generative model that regularizes latent space with sliced-Wasserstein distance, avoiding complex adversarial training.

Findings

SWAE effectively shapes latent distributions to match predefined distributions.

The method offers similar capabilities to WAE and VAE.

SWAE has an embarrassingly simple implementation.

Abstract

In this paper we study generative modeling via autoencoders while using the elegant geometric properties of the optimal transport (OT) problem and the Wasserstein distances. We introduce Sliced-Wasserstein Autoencoders (SWAE), which are generative models that enable one to shape the distribution of the latent space into any samplable probability distribution without the need for training an adversarial network or defining a closed-form for the distribution. In short, we regularize the autoencoder loss with the sliced-Wasserstein distance between the distribution of the encoded training samples and a predefined samplable distribution. We show that the proposed formulation has an efficient numerical solution that provides similar capabilities to Wasserstein Autoencoders (WAE) and Variational Autoencoders (VAE), while benefiting from an embarrassingly simple implementation.