Geometric Autoencoders -- What You See is What You Decode

TL;DR

This paper introduces Geometric Autoencoders, which incorporate differential geometry to improve visualization fidelity by reducing embedding distortion and providing diagnostics for more accurate data structure representation.

Contribution

It presents a novel geometric perspective on autoencoders and a regularizer that mitigates embedding distortion, enhancing visualization accuracy.

Findings

Reduces spurious stretching in embeddings

Provides diagnostics for embedding distortion

Improves faithful data visualization

Abstract

Visualization is a crucial step in exploratory data analysis. One possible approach is to train an autoencoder with low-dimensional latent space. Large network depth and width can help unfolding the data. However, such expressive networks can achieve low reconstruction error even when the latent representation is distorted. To avoid such misleading visualizations, we propose first a differential geometric perspective on the decoder, leading to insightful diagnostics for an embedding's distortion, and second a new regularizer mitigating such distortion. Our ``Geometric Autoencoder'' avoids stretching the embedding spuriously, so that the visualization captures the data structure more faithfully. It also flags areas where little distortion could not be achieved, thus guarding against misinterpretation.