TL;DR
MANTRA introduces a large-scale, diverse dataset of surface and 3D manifold triangulations to benchmark and evaluate higher-order topological deep learning models, highlighting their strengths and limitations.
Contribution
This paper provides the first extensive dataset for benchmarking higher-order models in topological deep learning, facilitating progress in the field.
Findings
Simplicial complex-based models outperform graph-based ones on simple topological invariants.
Current models struggle with complex topological features, indicating room for improvement.
MANTRA enables systematic evaluation of higher-order topological neural networks.
Abstract
The rising interest in leveraging higher-order interactions present in complex systems has led to a surge in more expressive models exploiting higher-order structures in the data, especially in topological deep learning (TDL), which designs neural networks on higher-order domains such as simplicial complexes. However, progress in this field is hindered by the scarcity of datasets for benchmarking these architectures. To address this gap, we introduce MANTRA, the first large-scale, diverse, and intrinsically higher-order dataset for benchmarking higher-order models, comprising over 43,000 and 250,000 triangulations of surfaces and three-dimensional manifolds, respectively. With MANTRA, we assess several graph- and simplicial complex-based models on three topological classification tasks. We demonstrate that while simplicial complex-based neural networks generally outperform their…
Peer Reviews
Decision·ICLR 2025 Poster
1. Extensive evaluation of currently available graph and simplicial complex-based models on MANTRA. 2. Provides a foundation for developing and benchmarking advanced TDL methods. 3. Insight into the ability of higher-order models to be invariant to triangulation transformations.
1. Evaluations are mostly focused on MP networks. It would help get better overall picture of the impact the dataset has by evaluating on other architectures like equivariant high-order neural nets and topological transformers. 2. The variance across results is surprising and I am not sure if running for more epochs/more hyperparameter tuning will address this.
- MANTRA is a large dataset specifically designed for topological property prediction, filling an important a gap in the TDL literature. Existing datasets are limited in scope, size, topological diversity and often rely on artificial topological lifting of graph data. - The paper presents systematic analysis of graph methods vs. topological methods on fundamental topological property prediction tasks, establishing important baselines for future research in TDL. - TDL models' ability to capture t
- **Topological diversity** - Other than the genus, first Betti number, and homeomorphism type for 2D manifolds, the distribution of topological invariants is limited and unbalanced. - For 3D manifolds: - Most Betti numbers, homeomorphism types, and torsion subgroups have only one or two possible values. - When two values do exist, one value represents over 99% of the cases. - All triangulations are limited to 10 or fewer vertices. - **Limited model coverage** - The only
The research question is interesting. It is novel to investigate the performance of graph and high-order network models on predicting topological features.
- There are various works on estimating Betti numbers, but the authors did not mention them. They should provide as solid baselines. For example: - Estimating Betti Numbers Using Deep Learning - A (simple) classical algorithm for estimating Betti numbers - The experiments are rather unclear, for example, details on how the architectures are not provided. As a paper in datasets and benchmarks, this is necessary and as a reviewer, I'm interested in knowing how the architectures are designed an
Code & Models
Videos
