Probing Graph Neural Network Activation Patterns Through Graph Topology

TL;DR

This paper investigates how graph topology influences GNN activation patterns, revealing that global attention mechanisms increase topological bottlenecks and that curvature can diagnose learning failures.

Contribution

It introduces Massive Activations as a probe and demonstrates how curvature shifts relate to GNN performance issues, providing new insights into topology-GNN interactions.

Findings

Massive Activations do not concentrate on curvature extremes.

Global attention increases negative curvature, worsening bottlenecks.

Curvature serves as a diagnostic for GNN learning failures.

Abstract

Curvature notions on graphs provide a theoretical description of graph topology, highlighting bottlenecks and denser connected regions. Artifacts of the message passing paradigm in Graph Neural Networks, such as oversmoothing and oversquashing, have been attributed to these regions. However, it remains unclear how the topology of a graph interacts with the learned preferences of GNNs. Through Massive Activations, which correspond to extreme edge activation values in Graph Transformers, we probe this correspondence. Our findings on synthetic graphs and molecular benchmarks reveal that MAs do not preferentially concentrate on curvature extremes, despite their theoretical link to information flow. On the Long Range Graph Benchmark, we identify a systemic \textit{curvature shift}: global attention mechanisms exacerbate topological bottlenecks, drastically increasing the prevalence of negative curvature. Our work reframes curvature as a diagnostic probe for understanding when and why graph learning fails.