Invertible Manifold Learning for Dimension Reduction

TL;DR

This paper introduces invertible manifold learning (inv-ML), a two-stage dimension reduction method that preserves topological and geometric properties of data manifolds, enabling lossless and efficient low-dimensional representations.

Contribution

The paper proposes a novel invertible manifold learning approach with a homeomorphic transformation and linear compression, bridging theoretical and practical dimension reduction.

Findings

inv-ML achieves invertible dimension reduction compared to existing methods.

The learned manifolds reveal key geometric characteristics through experiments.

Latent space interpolation demonstrates the importance of tangent space approximation.

Abstract

Dimension reduction (DR) aims to learn low-dimensional representations of high-dimensional data with the preservation of essential information. In the context of manifold learning, we define that the representation after information-lossless DR preserves the topological and geometric properties of data manifolds formally, and propose a novel two-stage DR method, called invertible manifold learning (inv-ML) to bridge the gap between theoretical information-lossless and practical DR. The first stage includes a homeomorphic sparse coordinate transformation to learn low-dimensional representations without destroying topology and a local isometry constraint to preserve local geometry. In the second stage, a linear compression is implemented for the trade-off between the target dimension and the incurred information loss in excessive DR scenarios. Experiments are conducted on seven datasets with a neural network implementation of inv-ML, called i-ML-Enc. Empirically, i-ML-Enc achieves invertible DR in comparison with typical existing methods as well as reveals the characteristics of the learned manifolds. Through latent space interpolation on real-world datasets, we find that the reliability of tangent space approximated by the local neighborhood is the key to the success of manifold-based DR algorithms.