On the Latent Space of Wasserstein Auto-Encoders

TL;DR

This paper investigates how the dimensionality of the latent space affects Wasserstein auto-encoders, advocating for random encoders and demonstrating their effectiveness in representation learning and disentanglement tasks.

Contribution

It introduces the idea that random encoders outperform deterministic ones in WAEs and explores their potential for improved representation learning.

Findings

Random encoders outperform deterministic encoders in WAEs

WAEs show promising results on disentanglement benchmarks

Latent space dimensionality significantly impacts WAE performance

Abstract

We study the role of latent space dimensionality in Wasserstein auto-encoders (WAEs). Through experimentation on synthetic and real datasets, we argue that random encoders should be preferred over deterministic encoders. We highlight the potential of WAEs for representation learning with promising results on a benchmark disentanglement task.