Thinner Latent Spaces: Detecting Dimension and Imposing Invariance with Conformal Autoencoders

TL;DR

This paper introduces a novel neural network architecture called Conformal Autoencoders that uses gradient orthogonality to determine intrinsic data dimensions, achieve disentanglement, and enforce invariance, with applications demonstrated on various datasets.

Contribution

It presents a new method leveraging differential geometry and gradient orthogonality in autoencoders to infer data manifold dimensions and impose invariance, advancing representation learning techniques.

Findings

Successfully infers intrinsic data dimensions

Achieves disentangled and invariant representations

Demonstrates applicability on multiple datasets

Abstract

Conformal Autoencoders are a neural network architecture that imposes orthogonality conditions between the gradients of latent variables to obtain disentangled representations of data. In this work we show that orthogonality relations within the latent layer of the network can be leveraged to infer the intrinsic dimensionality of nonlinear manifold data sets (locally characterized by the dimension of their tangent space), while simultaneously computing encoding and decoding (embedding) maps. We outline the relevant theory relying on differential geometry, and describe the corresponding gradient-descent optimization algorithm. The method is applied to several data sets and we highlight its applicability, advantages, and shortcomings. In addition, we demonstrate that the same computational technology can be used to build coordinate invariance to local group actions when defined only on a (reduced) submanifold of the embedding space.