What Are Good Positional Encodings for Directed Graphs?
Yinan Huang, Haoyu Wang, Pan Li
TL;DR
This paper introduces a novel positional encoding for directed graphs called Multi-q Magnetic Laplacian PE, which effectively captures structural features like walk profiles, enhancing neural network expressiveness and performance on circuit benchmarks.
Contribution
The work proposes a new PE method for directed graphs that can express walk profiles and adapts neural networks for stable use of this encoding, addressing a gap in existing research.
Findings
Proposed PE can express walk profiles.
Validated effectiveness on circuit benchmarks.
Improved neural network performance with new PE.
Abstract
Positional encodings (PEs) are essential for building powerful and expressive graph neural networks and graph transformers, as they effectively capture the relative spatial relationships between nodes. Although extensive research has been devoted to PEs in undirected graphs, PEs for directed graphs remain relatively unexplored. This work seeks to address this gap. We first introduce the notion of Walk Profile, a generalization of walk-counting sequences for directed graphs. A walk profile encompasses numerous structural features crucial for directed graph-relevant applications, such as program analysis and circuit performance prediction. We identify the limitations of existing PE methods in representing walk profiles and propose a novel Multi-q Magnetic Laplacian PE, which extends the Magnetic Laplacian eigenvector-based PE by incorporating multiple potential factors. The new PE can…
Peer Reviews
Decision·ICLR 2025 Poster
- This paper has an exemplary organization with clean definitions and explanation of fundamental developments and a balanced experimental section. - Both the notion of the walk profile and the generalization of Mag-PE to multiple q's are well motivated and simple to reason about. - It is important that the paper provides a provable connection of Multi-q Mag-PE (which is a latent representation) to something which is more "tangible": path type counters (i.e. walk profile) (Theorem 4.2).
- Runtime overhead, although quantified within the x3 envelope in Section 5.4, is a disadvantage in the approach. The combination of SPE and standard Mag-PE could strike a good balance (but then the "which q value" question will be raised). Also depending on the number of q's, runtime overhead would vary. On this front, having a rule of thumb to adopt for the cardinality of q's (i.e. Q) for a given task/dataset combination, would be very useful to have.
- The theoretical arguments made by the paper are sound and well-thought: The definition and accompanying figures for walk profiles are very helpful, and are very well connected to real-world settings. Moreover, the proof sketch for showing how Multi-q Magnetic Laplacian can recover walk profiles in Theorem 4.2 appears sound. Overall, the flow of argumentation in this paper is smooth, balanced, and easy to follow, with all pre-requisite results presented in good time. - The experimental analysis
- One aspect of this work that could be improved is to include baselines and datasets used in previous work, such as OGBG-code, NA and BN (as used, e.g., in Thost et al). The current results are quite strong, but I imagine that the message would be even more compelling if your findings also apply to more specialized models, and produce results on the corresponding datasets to glean more interesting insights, e.g., can these PEs even improve specialized directed GNN models? In general, I find t
- The paper is presented adequately, and is easy to read. - The interest in being able to express the walk profiles is nicely motivated in the introduction. - Walk profiles are introduced as a way of counting walks which is more comprehensive than simple shortest path distances, which can be trivial in directed graphs. - The authors show empirically that their method is superior when simple models are used on simple tasks. They also show that theoretically their PE is more expressive with respec
- As the authors have discussed it in Section 4.3, the main limitation is the extra compute-time with larger $L$, induced namely by the calculation of large and numerous eigendecompositions. As shown in Section 5.4, the cost seems to grow linearly. - While it is described informally why walk profiles can be of interest, there is no theoretical justification of their use. Moreover, there are no connections to existing measures of modelsʼ expressivity (e.g., $k$-WL[1]). - It is not clear whether t
Code & Models
Videos
Taxonomy
TopicsDNA and Biological Computing · Genome Rearrangement Algorithms · Algorithms and Data Compression
MethodsLaplacian EigenMap · Laplacian Positional Encodings