Large-Scale Spectral Graph Neural Networks via Laplacian Sparsification: Technical Report

TL;DR

This paper introduces SGNN-LS, a spectral graph neural network method using Laplacian sparsification to efficiently scale spectral GNNs to large graphs while enabling end-to-end training with raw features.

Contribution

The paper proposes a novel Laplacian sparsification technique for spectral GNNs that preserves polynomial filter properties and allows end-to-end training on large-scale graphs.

Findings

Outperforms existing spectral GNN approximations on large datasets

Enables end-to-end training with raw node features

Achieves superior efficiency and effectiveness on billion-scale graphs

Abstract

Graph Neural Networks (GNNs) play a pivotal role in graph-based tasks for their proficiency in representation learning. Among the various GNN methods, spectral GNNs employing polynomial filters have shown promising performance on tasks involving both homophilous and heterophilous graph structures. However, The scalability of spectral GNNs on large graphs is limited because they learn the polynomial coefficients through multiple forward propagation executions during forward propagation. Existing works have attempted to scale up spectral GNNs by eliminating the linear layers on the input node features, a change that can disrupt end-to-end training, potentially impact performance, and become impractical with high-dimensional input features. To address the above challenges, we propose "Spectral Graph Neural Networks with Laplacian Sparsification (SGNN-LS)", a novel graph spectral sparsification method to approximate the propagation patterns of spectral GNNs. We prove that our proposed method generates Laplacian sparsifiers that can approximate both fixed and learnable polynomial filters with theoretical guarantees. Our method allows the application of linear layers on the input node features, enabling end-to-end training as well as the handling of raw text features. We conduct an extensive experimental analysis on datasets spanning various graph scales and properties to demonstrate the superior efficiency and effectiveness of our method. The results show that our method yields superior results in comparison with the corresponding approximated base models, especially on dataset Ogbn-papers100M(111M nodes, 1.6B edges) and MAG-scholar-C (2.8M features).