Autoencoding Dynamics: Topological Limitations and Capabilities

TL;DR

This paper explores the topological constraints and possibilities of autoencoders in representing data manifolds and their ability to encode dynamical systems with invariant manifolds.

Contribution

It provides a topological analysis of autoencoder limitations and capabilities, especially for dynamical systems with invariant manifolds.

Findings

Identifies topological obstructions to autoencoding certain manifolds.

Describes conditions under which autoencoders can successfully encode dynamical systems.

Highlights the role of topology in autoencoder design and capabilities.

Abstract

Given a "data manifold" $M\subset \mathbb{R}^n$ and "latent space" $\mathbb{R}^\ell$, an autoencoder is a pair of continuous maps consisting of an "encoder" $E\colon \mathbb{R}^n\to \mathbb{R}^\ell$ and "decoder" $D\colon \mathbb{R}^\ell\to \mathbb{R}^n$ such that the "round trip" map $D\circ E$ is as close as possible to the identity map $\mbox{id}_M$ on $M$. We present various topological limitations and capabilites inherent to the search for an autoencoder, and describe capabilities for autoencoding dynamical systems having $M$ as an invariant manifold.