Manifold-Matching Autoencoders
Laurent Cheret, Vincent L\'etourneau, Isar Nejadgholi, Chris Drummond, Hussein Al Osman, Maia Fraser
TL;DR
Manifold-Matching Autoencoders (MMAE) introduce an unsupervised regularization that aligns pairwise distances in latent and input spaces, improving data representation quality and approximating Multi-Dimensional Scaling.
Contribution
This paper proposes MMAE, a novel regularization method for autoencoders that aligns pairwise distances, enhancing representation quality and scalability.
Findings
Outperforms similar methods on nearest-neighbor metrics
Effective in preserving topological data features
Provides a scalable approximation of MDS
Abstract
We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error. Because alignment occurs on pairwise distances rather than coordinates, it can also be extended to a lower-dimensional representation of the data, adding flexibility to the method. We find that this regularization outperforms similar methods on metrics based on preservation of nearest-neighbor distances and persistent homology-based measures. We also observe that MMAE provides a scalable approximation of Multi-Dimensional Scaling (MDS).
Peer Reviews
Decision·ICLR 2026 Conference Withdrawn Submission
- The approach is scale-invariant and does not require the latent dimensionality to equal the intrinsic manifold dimension, potentially simplifying autoencoder design and enabling high-dimensional latents. - It is intriguing that geometry preservation appears to improve semantic interpolation quality, although the underlying rationale remains unclear.
- **Limited novelty.** Similar ideas that preserve the geometry of embeddings/graphs already exist. Direct comparisons are missing, e.g. - SPAE: Singh & Nag (2021), Structure-preserving deep autoencoder-based dimensionality reduction for data visualization, IEEE SNPD. - GGAE: Lim, Kim, Lee, Jang & Park (2024), Graph Geometry-Preserving Autoencoders, ICML. The paper needs to position MMAE against these and related geometry/graph-preserving autoencoders with clear conceptual and empirical d
This paper originates from a good motivation: preserving the data topology in the latent space. Topological data analysis (TDA) approaches are either too expensive in computation or too narrowly applicable (e.g., to only 2D/3D visualization). The proposed method addresses the drawbacks of TDA by regularizing an autoencoder, a simple and elegant approach. However, despite its simplicity, the paper has not done enough theoretical/empirical exploration (see the Weakness section below). The authors
Despite a good motivation, the paper leaves more to be desired. The theoretical foundation is weak. The proposed distance regularization looks ad hoc. It is unclear why the authors propose such a regularization among many obvious alternatives. It might be helpful if the authors could set the stage with more background information on topology/manifold and topological data analysis, as justification for the proposed method. A weakness of the method is that it leaves open what a good reference e
- **Text**: The paper is nicely structured and easy to follow. The idea is explained in detail. - **Experiments**: two setups are explored -- generative (VAE) and NLDR.
- **Idea**: The method depends entirely on precomputed embeddings (e.g., PCA, UMAP), which already provide a geometric structure. It is not clearly explained why training a parametric autoencoder is required beyond simply using these embeddings directly. The paper argues that topological autoencoder methods fail in higher-dimensional bottlenecks due to the computational costs of persistence homology; however, just like MMAE, it also operates on the pairwise distances and doesn't depend on the di
The paper's organization is mostly fine, the main ideas are easy to follow.
1) The main weakness is a limited scientific novelty. In a nutshell, authors propose to add a distance-based regularizer between real-latent spaces to an AE objective. 2) Experimental results in Fig. 3 are very hard to interpret. Consider presenting results in a Table or by a different visualization. Comparison with SOTA methods like IVIS, RTD-AE, PacMAP, PHATE, and MDS are missing. Consider evaluating linear correlation, the triplet distance ranking accuracy, Wasserstein distance between pers
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopological and Geometric Data Analysis · 3D Shape Modeling and Analysis · Advanced Graph Neural Networks