Breaking the Limits of Message Passing Graph Neural Networks

TL;DR

This paper introduces a spectral-domain convolutional approach for message passing graph neural networks that surpasses the expressive power of traditional models, achieving 3-WL equivalence with improved computational efficiency and spectral richness.

Contribution

The paper proposes a novel spectral filter design that enhances GNN expressive power beyond 1-WL, matching 3-WL capabilities while maintaining spatial localization and computational efficiency.

Findings

Achieves 3-WL expressive power with spectral filters.

Outperforms existing models in downstream tasks.

Reduces computational complexity compared to 3-WL models.

Abstract

Since the Message Passing (Graph) Neural Networks (MPNNs) have a linear complexity with respect to the number of nodes when applied to sparse graphs, they have been widely implemented and still raise a lot of interest even though their theoretical expressive power is limited to the first order Weisfeiler-Lehman test (1-WL). In this paper, we show that if the graph convolution supports are designed in spectral-domain by a non-linear custom function of eigenvalues and masked with an arbitrary large receptive field, the MPNN is theoretically more powerful than the 1-WL test and experimentally as powerful as a 3-WL existing models, while remaining spatially localized. Moreover, by designing custom filter functions, outputs can have various frequency components that allow the convolution process to learn different relationships between a given input graph signal and its associated properties. So far, the best 3-WL equivalent graph neural networks have a computational complexity in $\mathcal{O}(n^3)$ with memory usage in $\mathcal{O}(n^2)$, consider non-local update mechanism and do not provide the spectral richness of output profile. The proposed method overcomes all these aforementioned problems and reaches state-of-the-art results in many downstream tasks.