On the Completeness of Invariant Geometric Deep Learning Models
Zian Li, Xiyuan Wang, Shijia Kang, Muhan Zhang
TL;DR
This paper rigorously analyzes the expressive power of invariant geometric deep learning models, demonstrating that several key models can achieve E(3)-completeness, thus advancing theoretical understanding of their capabilities.
Contribution
It provides the first comprehensive theoretical characterization of the expressiveness of invariant models, including proving E(3)-completeness for GeoNGNN and other established models.
Findings
DisGNN is limited to highly symmetric point clouds.
GeoNGNN achieves E(3)-completeness, breaking symmetry restrictions.
DimeNet, GemNet, SphereNet also attain E(3)-completeness.
Abstract
Invariant models, one important class of geometric deep learning models, are capable of generating meaningful geometric representations by leveraging informative geometric features in point clouds. These models are characterized by their simplicity, good experimental results and computational efficiency. However, their theoretical expressive power still remains unclear, restricting a deeper understanding of the potential of such models. In this work, we concentrate on characterizing the theoretical expressiveness of a wide range of invariant models under fully-connected conditions. We first rigorously characterize the expressiveness of the most classic invariant model, message-passing neural networks incorporating distance (DisGNN), restricting its unidentifiable cases to be only highly symmetric point clouds. We then prove that GeoNGNN, the geometric counterpart of one of the simplest…
Peer Reviews
Decision·ICLR 2025 Poster
- The paper attempts to address a crucial problem that enhances our understanding of the potential of invariant neural networks and can guide future model design. - Investigating the geometric counterparts of subgraph GNNs is a novel contribution. - The results extend beyond specific cases, such as asymmetric point clouds, broadening our understanding of how these models perform on symmetric point clouds as well.
- The paper lacks clarity and structure in some areas. The detailed explanation of NGNN, which serves as the backbone of their main contribution, the GeoNGNN framework, is left in the appendix. I recommend the authors integrate key aspects of NGNN, such as its core equations or an architectural diagram, into the main text. Additionally, including a comparison with the original DisGNN would be helpful—highlighting the differences and explaining what enables NGNN (intuitively) to overcome the limi
The paper introduces a novel conceptual framework for understanding the efficacy of certain invariant architectures. This is further supported through theoretical analysis and empirical studies. Additionally, it proposes a framework for the development of future architectures extending the impact and significance of the work.
The paper lacks sufficient empirical evidence to support its theoretical analysis, significantly reducing the overall significance and impact of the work. The selected real world experiments emphasize datasets which lack conformers or nearly isomorphic point clouds. Furthermore, the main text does not provide adequate evidence to demonstrate the advantages of GeoNGNN over the existing complete invariant architectures. Additionally, the excessive use of bold text and the absence of a clear outli
The authors have provided both extensive proofs as well as extensive analysis to their claims. Overall the presentation and intend is clear and definitions are well-thought out and the authors provide a good heuristic insight with each introduced theorem and definition which is nice. The extensive analysis of both DisGNN and GeoNGNN shows that the work is of good quality and looks to be of good quality to the reviewer. All theorems come with extensive proofs and with a intuition which is helpful
To the reader it seems that some of the definitions are somewhat convolved and some simplification and clarity in the definitions might improve reading. Some of the definitions, while they might be customary in the machine learning literature, are somewhat unfortunately choses from a mathematical perspective. Completeness of a space in the mathematical sense implies that each Cauchy sequence has a limit within that space. A second example is the use of the term isomorphism. While not wrong, a be
Code & Models
Videos
Taxonomy
TopicsGeological Modeling and Analysis · Image Processing and 3D Reconstruction