Learning low bending and low distortion manifold embeddings

TL;DR

This paper introduces a regularization method for autoencoders that promotes smooth, isometric, and flat manifold embeddings in latent space, improving interpolation and data representation quality.

Contribution

It proposes a novel loss functional based on local distances and Frechet averages to regularize autoencoder embeddings, enabling training without additional data.

Findings

Embeddings become smoother and more isometric.

Linear interpolation in latent space approximates manifold interpolation.

Numerical tests confirm improved manifold regularity.

Abstract

Autoencoders are a widespread tool in machine learning to transform high-dimensional data into a lowerdimensional representation which still exhibits the essential characteristics of the input. The encoder provides an embedding from the input data manifold into a latent space which may then be used for further processing. For instance, learning interpolation on the manifold may be simplified via the new manifold representation in latent space. The efficiency of such further processing heavily depends on the regularity and structure of the embedding. In this article, the embedding into latent space is regularized via a loss function that promotes an as isometric and as flat embedding as possible. The required training data comprises pairs of nearby points on the input manifold together with their local distance and their local Frechet average. This regularity loss functional even allows to train the encoder on its own. The loss functional is computed via a Monte Carlo integration which is shown to be consistent with a geometric loss functional defined directly on the embedding map. Numerical tests are performed using image data that encodes different data manifolds. The results show that smooth manifold embeddings in latent space are obtained. These embeddings are regular enough such that interpolation between not too distant points on the manifold is well approximated by linear interpolation in latent space.