How does over-squashing affect the power of GNNs?

TL;DR

This paper analyzes how over-squashing limits the expressive power of Message Passing Neural Networks (MPNNs) in graph learning, providing theoretical insights and experimental validation on the impact of graph structure and network capacity.

Contribution

It introduces a novel measure for over-squashing, linking it to the expressive limitations of MPNNs based on graph properties and network capacity.

Findings

Over-squashing restricts pairwise node communication in MPNNs.

Larger network capacity can mitigate over-squashing effects.

Theoretical impossibility results highlight fundamental limits of MPNNs.

Abstract

Graph Neural Networks (GNNs) are the state-of-the-art model for machine learning on graph-structured data. The most popular class of GNNs operate by exchanging information between adjacent nodes, and are known as Message Passing Neural Networks (MPNNs). Given their widespread use, understanding the expressive power of MPNNs is a key question. However, existing results typically consider settings with uninformative node features. In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity. We do so by measuring the level of pairwise interactions between nodes that MPNNs allow for. This measure provides a novel quantitative characterization of the so-called over-squashing effect, which is observed to occur when a large volume of messages is aggregated into fixed-size vectors. Using our measure, we prove that, to guarantee sufficient communication between pairs of nodes, the capacity of the MPNN must be large enough, depending on properties of the input graph structure, such as commute times. For many relevant scenarios, our analysis results in impossibility statements in practice, showing that over-squashing hinders the expressive power of MPNNs. We validate our theoretical findings through extensive controlled experiments and ablation studies.