Cayley Graph Propagation

TL;DR

This paper introduces CGP, a method for propagating information over complete Cayley graphs to address over-squashing in graph neural networks, showing improved performance over previous expander graph approaches and graph rewiring techniques.

Contribution

Proposes CGP, a novel approach that uses complete Cayley graphs for bottleneck-free information propagation in GNNs, improving over truncated Cayley graph methods.

Findings

CGP significantly improves information flow in GNNs.

CGP outperforms or matches complex graph rewiring techniques.

Empirical results demonstrate better or comparable performance across datasets.

Abstract

In spite of the plethora of success stories with graph neural networks (GNNs) on modelling graph-structured data, they are notoriously vulnerable to over-squashing, whereby tasks necessitate the mixing of information between distance pairs of nodes. To address this problem, prior work suggests rewiring the graph structure to improve information flow. Alternatively, a significant body of research has dedicated itself to discovering and precomputing bottleneck-free graph structures to ameliorate over-squashing. One well regarded family of bottleneck-free graphs within the mathematical community are expander graphs, with prior work -- Expander Graph Propagation (EGP) -- proposing the use of a well-known expander graph family -- the Cayley graphs of the $\mathrm{SL}(2,\mathbb{Z}_n)$ special linear group -- as a computational template for GNNs. However, in EGP the computational graphs used are truncated to align with a given input graph. In this work, we show that truncation is detrimental to the coveted expansion properties. Instead, we propose CGP, a method to propagate information over a complete Cayley graph structure, thereby ensuring it is bottleneck-free to better alleviate over-squashing. Our empirical evidence across several real-world datasets not only shows that CGP recovers significant improvements as compared to EGP, but it is also akin to or outperforms computationally complex graph rewiring techniques.