Representation Learning via Manifold Flattening and Reconstruction

TL;DR

This paper introduces FlatNet, a neural network approach that explicitly linearizes and reconstructs embedded manifolds, offering interpretability, scalability, and strong generalization, demonstrated on synthetic and image data.

Contribution

It presents a novel neural network architecture, FlatNet, that explicitly flattens and reconstructs manifolds, balancing interpretability, scalability, and generalization.

Findings

FlatNet effectively linearizes and reconstructs manifolds.

It outperforms other models on synthetic and image datasets.

The approach is computationally feasible and interpretable.

Abstract

This work proposes an algorithm for explicitly constructing a pair of neural networks that linearize and reconstruct an embedded submanifold, from finite samples of this manifold. Our such-generated neural networks, called Flattening Networks (FlatNet), are theoretically interpretable, computationally feasible at scale, and generalize well to test data, a balance not typically found in manifold-based learning methods. We present empirical results and comparisons to other models on synthetic high-dimensional manifold data and 2D image data. Our code is publicly available.