Hyperspherical Variational Auto-Encoders

TL;DR

This paper introduces hyperspherical Variational Auto-Encoders using von Mises-Fisher distributions to better model data with latent hyperspherical structures, outperforming traditional Gaussian VAEs in certain scenarios.

Contribution

It proposes a novel hyperspherical VAE framework with von Mises-Fisher distributions, addressing limitations of Gaussian assumptions for data with spherical latent structures.

Findings

Hyperspherical VAEs better capture data with spherical latent structures.

Hyperspherical VAEs outperform Gaussian VAEs in low-dimensional settings.

The approach is effective for various data types with hyperspherical characteristics.

Abstract

The Variational Auto-Encoder (VAE) is one of the most used unsupervised machine learning models. But although the default choice of a Gaussian distribution for both the prior and posterior represents a mathematically convenient distribution often leading to competitive results, we show that this parameterization fails to model data with a latent hyperspherical structure. To address this issue we propose using a von Mises-Fisher (vMF) distribution instead, leading to a hyperspherical latent space. Through a series of experiments we show how such a hyperspherical VAE, or $\mathcal{S}$-VAE, is more suitable for capturing data with a hyperspherical latent structure, while outperforming a normal, $\mathcal{N}$-VAE, in low dimensions on other data types. Code at http://github.com/nicola-decao/s-vae-tf and https://github.com/nicola-decao/s-vae-pytorch