Temporal Graph Rewiring with Expander Graphs
Katarina Petrovi\'c, Shenyang Huang, Farimah Poursafaei, Petar, Veli\v{c}kovi\'c
TL;DR
This paper introduces Temporal Graph Rewiring (TGR), a novel method for improving temporal graph neural networks by constructing message passing pathways between distant nodes, leading to state-of-the-art results on multiple benchmarks.
Contribution
TGR is the first approach to apply graph rewiring to temporal graphs, enhancing message passing and addressing issues like under-reaching and over-squashing.
Findings
TGR achieves state-of-the-art results on TGB benchmark datasets.
50.5% improvement in MRR on tgbl-review over base TGN.
22.2% improvement in MRR on tgbl-review over base TNCN.
Abstract
Evolving relations in real-world networks are often modelled by temporal graphs. Temporal Graph Neural Networks (TGNNs) emerged to model evolutionary behaviour of such graphs by leveraging the message passing primitive at the core of Graph Neural Networks (GNNs). It is well-known that GNNs are vulnerable to several issues directly related to the input graph topology, such as under-reaching and over-squashing - we argue that these issues can often get exacerbated in temporal graphs, particularly as the result of stale nodes and edges. While graph rewiring techniques have seen frequent usage in GNNs to make the graph topology more favourable for message passing, they have not seen any mainstream usage on TGNNs. In this work, we propose Temporal Graph Rewiring (TGR), the first approach for graph rewiring on temporal graphs, to the best of our knowledge. TGR constructs message passing…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
- This paper provides solid theoretical motivation concerning how under-reaching and over-squashing issues are exacerbated in the context of dynamic graph problems. - The experiments are thorough on the Temporal Graph Benchmark (TGB). The datasets in TGB appear to be large-scale, providing sufficient experimental justification for the proposed techniques. - The overall presentation of the work is good, and the designed method is easy to understand.
- The paper is directly built on the existing work of Deac et al. (2022), extending it to a new dynamic graph setting. This integration appears straightforward as the use of expander graphs is not specifically tailored for temporal graphs. - The authors seem to believe that the only way to address the issues of under-reaching and over-squashing is through graph rewiring using expander graphs. However, under-reaching and over-squashing are commonly acknowledged problems in graph theory, and ther
- The paper sheds light on an interesting and seemingly relevant phenomenon; the community may likely benefit from the explicit formalisation of this problem as the authors set to do in this manuscript; - The discussions generalising the analysis to consider practical implementation and training details is interesting and useful; - The approach proposed to alleviate the highlighted problem is simple and effective, other than already known and studied by the community in other contexts.
- In the current form, the manuscript is not optimally presented and is not making effective use of the space available in the main paper. In particular, the presentation could be largely improved by: - Adding more exemplary illustrations to visually support the analyses and results on the temporal under-reaching phenomenon; - Relegating to appendix subsections illustrating concepts related to static graphs; at the moment they are seemingly presented in more detail than those on dynamic
- The under-reaching and over-squashing phenomenon over temporal graphs is a nascent research field. The paper provide insights that such issues might be overlooked in previous studies on temporal graph representation learning. - The proposed TGR framework can serve as a backbone-agnostic augmentation to any off-the-shelf temporal graph neural models, which is flexible with strong empirical performance.
- Limited novelty: The main algorithmic contribution of the paper, TGR, is an ad-hoc combination of the EGP method [1] with standard temporal GNN protocols. - Lack of empirical evidence to under-reaching in temporal graphs: In section 3.3 of the paper, the authors state that the under-reaching effect in temporal graphs might be more severe than that in static graphs. While the authors present two propositions, the constructions therein are contrived and not realistic. It is desirable that the a
+ The writing is clear, and the paper is well-organized. + The approach is interesting and addresses an important topic. + Code for reproducing the experiments is provided.
+ The proposed methodology is limited to continuous-time temporal graphs, and further work would be needed to extend the model to discrete-time settings. + The experiments are conducted on a temporal link prediction task, which is insufficient to fully validate the performance of TGR. It is suggested to include node-level tasks, such as dynamic node classification, to demonstrate the broader applicability of TGR. + Although TGR is proposed to address under-reaching and over-squashing, there are
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGraph Theory and Algorithms · Complex Network Analysis Techniques
MethodsBalanced Selection · Temporal Graph Network