Spatio-Spectral Graph Neural Networks

TL;DR

This paper introduces Spatio-Spectral Graph Neural Networks (S$^2$GNNs), a novel approach that combines spatial and spectral graph filters to overcome limitations of traditional MPGNNs, enabling more expressive and scalable graph learning.

Contribution

The paper proposes S$^2$GNNs, a new GNN paradigm that integrates spatial and spectral filters, improving information propagation, expressiveness, and scalability over existing methods.

Findings

S$^2$GNNs mitigate over-squashing and provide tighter approximation bounds.

They enable more expressive models than the 1-WL test with positional encodings.

S$^2$GNNs outperform existing methods on peptide benchmarks and scale to large graphs.

Abstract

Spatial Message Passing Graph Neural Networks (MPGNNs) are widely used for learning on graph-structured data. However, key limitations of l-step MPGNNs are that their "receptive field" is typically limited to the l-hop neighborhood of a node and that information exchange between distant nodes is limited by over-squashing. Motivated by these limitations, we propose Spatio-Spectral Graph Neural Networks (S$^2$GNNs) -- a new modeling paradigm for Graph Neural Networks (GNNs) that synergistically combines spatially and spectrally parametrized graph filters. Parameterizing filters partially in the frequency domain enables global yet efficient information propagation. We show that S$^2$GNNs vanquish over-squashing and yield strictly tighter approximation-theoretic error bounds than MPGNNs. Further, rethinking graph convolutions at a fundamental level unlocks new design spaces. For example, S$^2$GNNs allow for free positional encodings that make them strictly more expressive than the 1-Weisfeiler-Lehman (WL) test. Moreover, to obtain general-purpose S$^2$GNNs, we propose spectrally parametrized filters for directed graphs. S$^2$GNNs outperform spatial MPGNNs, graph transformers, and graph rewirings, e.g., on the peptide long-range benchmark tasks, and are competitive with state-of-the-art sequence modeling. On a 40 GB GPU, S$^2$GNNs scale to millions of nodes.