Effective Structural Encodings via Local Curvature Profiles
Lukas Fesser, Melanie Weber
TL;DR
This paper introduces a novel graph structural encoding based on discrete Ricci curvature, demonstrating it outperforms existing methods and enhances GNN performance when combined with positional encodings, surpassing rewiring techniques.
Contribution
The paper proposes Local Curvature Profiles (LCP) as a new geometric structural encoding for GNNs, showing its superiority over existing encodings and effectiveness when combined with positional information.
Findings
LCP significantly outperforms existing structural encodings.
Combining LCP with positional encodings improves downstream tasks.
Curvature-based rewiring is less effective than LCP for performance enhancement.
Abstract
Structural and Positional Encodings can significantly improve the performance of Graph Neural Networks in downstream tasks. Recent literature has begun to systematically investigate differences in the structural properties that these approaches encode, as well as performance trade-offs between them. However, the question of which structural properties yield the most effective encoding remains open. In this paper, we investigate this question from a geometric perspective. We propose a novel structural encoding based on discrete Ricci curvature (Local Curvature Profiles, short LCP) and show that it significantly outperforms existing encoding approaches. We further show that combining local structural encodings, such as LCP, with global positional encodings improves downstream performance, suggesting that they capture complementary geometric information. Finally, we compare different…
Peer Reviews
Decision·ICLR 2024 poster
- LCP provides a unique way to encode the geometry of a node's neighborhood, and the paper convincingly demonstrates its superior performance in node and graph classification tasks. - The paper investigates the combination of local structural encodings with global positional encodings, showing that they capture complementary information about the graph. This finding is valuable as it suggests that using a combination of different encoding types can result in enhanced downstream performance. The
- Some parts of the introduction are a bit dense and may be challenging for readers not deeply familiar with the field. A clearer presentation of the background and motivation could benefit a wider audience. - Including experiments on a more diverse set of datasets and domains would be better.
This paper seems reasonable to this reviewer, outperforming baseline encoding approaches or approaches that require rewiring. Perhaps most surprising to this reviewer is that it improves performance of GATs (seemingly similar to transforms in that they use self-attention?) as it would seem reasonable that such a network would be able to dynamically compute something similar to these statistics. The experiments seem reasonably done at least to this reviewer (not an expert in this area at all),
It's not obvious to this reviewer what the weaknesses are. The main concern to this reviewer is that some large pretrained transformer could do better than any of the proposed methods, but that's a very general concern these days. Possibly this approach or GNNs in general could work better on more specialized tasks where there are a very large number of nodes.
The introduction of the local curvatures for structural encoding in graph neural networks is the key contribution of the paper. A theoretical result (Theorem 1) is also established suggesting improved expressivity due to LCP. However the result is rather qualitative without a quantitative characterization of the extent to which the expressivity is improved. Thus the theoretical development is rather light. Overall the paper is very well written and, for most parts, easy to read. The idea is
I do not see an obvious weakness in the paper, just like I do not see its development particularly striking. To me, the paper falls into those works that have a sound intuitive idea, which is validated via empirical evaluation. The paper does not appear to touch on the studied problem (i.e., the issues of over-smoothing and over-squashing) at a fundamental level or at depth. But it is perhaps above the acceptance threshold.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMedical Imaging and Analysis · Advanced Graph Neural Networks · Brain Tumor Detection and Classification