On Oversquashing in Graph Neural Networks Through the Lens of Dynamical Systems

TL;DR

This paper introduces SWAN, a novel GNN model inspired by dynamical systems, which effectively mitigates oversquashing by maintaining constant information flow, especially over long distances, validated through theoretical and empirical analysis.

Contribution

The paper proposes SWAN, a parameterized GNN with antisymmetry properties, to address oversquashing by leveraging dynamical systems concepts for improved long-range information transmission.

Findings

SWAN effectively mitigates oversquashing in GNNs.

Theoretical analysis confirms enhanced long-distance information flow.

Empirical results on benchmarks validate SWAN's performance improvements.

Abstract

A common problem in Message-Passing Neural Networks is oversquashing -- the limited ability to facilitate effective information flow between distant nodes. Oversquashing is attributed to the exponential decay in information transmission as node distances increase. This paper introduces a novel perspective to address oversquashing, leveraging dynamical systems properties of global and local non-dissipativity, that enable the maintenance of a constant information flow rate. We present SWAN, a uniquely parameterized GNN model with antisymmetry both in space and weight domains, as a means to obtain non-dissipativity. Our theoretical analysis asserts that by implementing these properties, SWAN offers an enhanced ability to transmit information over extended distances. Empirical evaluations on synthetic and real-world benchmarks that emphasize long-range interactions validate the theoretical understanding of SWAN, and its ability to mitigate oversquashing.