Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

TL;DR

This paper revisits spectral graph convolutional networks, identifies issues with ChebNet's approximation, and introduces ChebNetII, a new model that improves approximation and achieves superior performance on large-scale graph datasets.

Contribution

The paper introduces ChebNetII, an improved Chebyshev-based GNN that reduces overfitting and the Runge phenomenon, demonstrating superior performance on large-scale graph data.

Findings

ChebNet's poor performance is due to illegal coefficients causing overfitting.

ChebNetII outperforms previous models in node classification tasks.

ChebNetII scales effectively to billion-scale graphs like ogbn-papers100M.

Abstract

Designing spectral convolutional networks is a challenging problem in graph learning. ChebNet, one of the early attempts, approximates the spectral graph convolutions using Chebyshev polynomials. GCN simplifies ChebNet by utilizing only the first two Chebyshev polynomials while still outperforming it on real-world datasets. GPR-GNN and BernNet demonstrate that the Monomial and Bernstein bases also outperform the Chebyshev basis in terms of learning the spectral graph convolutions. Such conclusions are counter-intuitive in the field of approximation theory, where it is established that the Chebyshev polynomial achieves the optimum convergent rate for approximating a function. In this paper, we revisit the problem of approximating the spectral graph convolutions with Chebyshev polynomials. We show that ChebNet's inferior performance is primarily due to illegal coefficients learnt by ChebNet approximating analytic filter functions, which leads to over-fitting. We then propose ChebNetII, a new GNN model based on Chebyshev interpolation, which enhances the original Chebyshev polynomial approximation while reducing the Runge phenomenon. We conducted an extensive experimental study to demonstrate that ChebNetII can learn arbitrary graph convolutions and achieve superior performance in both full- and semi-supervised node classification tasks. Most notably, we scale ChebNetII to a billion graph ogbn-papers100M, showing that spectral-based GNNs have superior performance. Our code is available at https://github.com/ivam-he/ChebNetII.