Return of ChebNet: Understanding and Improving an Overlooked GNN on Long Range Tasks

TL;DR

This paper revisits ChebNet, an early spectral GNN, demonstrating its competitive performance on long-range tasks, addressing training stability issues with a new stable dynamical system formulation, and achieving near state-of-the-art results.

Contribution

The paper introduces Stable-ChebNet, a stable and scalable variant of ChebNet that effectively models long-range dependencies without complex rewiring or positional encodings.

Findings

ChebNet shows competitive advantages on long-range benchmarks.

Unstable training issues are addressed with a new dynamical system formulation.

Stable-ChebNet achieves near state-of-the-art performance across benchmarks.

Abstract

ChebNet, one of the earliest spectral GNNs, has largely been overshadowed by Message Passing Neural Networks (MPNNs), which gained popularity for their simplicity and effectiveness in capturing local graph structure. Despite their success, MPNNs are limited in their ability to capture long-range dependencies between nodes. This has led researchers to adapt MPNNs through rewiring or make use of Graph Transformers, which compromises the computational efficiency that characterized early spatial message-passing architectures, and typically disregards the graph structure. Almost a decade after its original introduction, we revisit ChebNet to shed light on its ability to model distant node interactions. We find that out-of-box, ChebNet already shows competitive advantages relative to classical MPNNs and GTs on long-range benchmarks, while maintaining good scalability properties for high-order polynomials. However, we uncover that this polynomial expansion leads ChebNet to an unstable regime during training. To address this limitation, we cast ChebNet as a stable and non-dissipative dynamical system, which we coin Stable-ChebNet. Our Stable-ChebNet model allows for stable information propagation, and has controllable dynamics which do not require the use of eigendecompositions, positional encodings, or graph rewiring. Across several benchmarks, Stable-ChebNet achieves near state-of-the-art performance.