A Spectral Analysis of Graph Neural Networks on Dense and Sparse Graphs

TL;DR

This paper investigates how graph sparsity influences the spectral properties and performance of graph neural networks (GNNs) in node classification, revealing that GNNs can outperform spectral methods on sparse graphs.

Contribution

It introduces a new random graph model to study sparsity effects and compares GNNs with spectral methods, highlighting their relative strengths across different graph densities.

Findings

GNNs outperform spectral methods on sparse graphs

Spectral properties vary with graph sparsity

Numerical experiments confirm theoretical insights

Abstract

In this work we propose a random graph model that can produce graphs at different levels of sparsity. We analyze how sparsity affects the graph spectra, and thus the performance of graph neural networks (GNNs) in node classification on dense and sparse graphs. We compare GNNs with spectral methods known to provide consistent estimators for community detection on dense graphs, a closely related task. We show that GNNs can outperform spectral methods on sparse graphs, and illustrate these results with numerical examples on both synthetic and real graphs.