Revisiting Graph Neural Networks: All We Have is Low-Pass Filters

TL;DR

This paper analyzes graph neural networks through the lens of graph signal processing, revealing they primarily perform low-pass filtering and lack non-linear manifold learning, which impacts their design and robustness.

Contribution

It introduces a theoretical framework showing GNNs act as low-pass filters and lack certain non-linear capabilities, providing new insights into their functioning and design.

Findings

GNNs mainly perform low-pass filtering on features.

They do not exhibit non-linear manifold learning.

GNNs' performance is influenced by the informativeness of initial features.

Abstract

Graph neural networks have become one of the most important techniques to solve machine learning problems on graph-structured data. Recent work on vertex classification proposed deep and distributed learning models to achieve high performance and scalability. However, we find that the feature vectors of benchmark datasets are already quite informative for the classification task, and the graph structure only provides a means to denoise the data. In this paper, we develop a theoretical framework based on graph signal processing for analyzing graph neural networks. Our results indicate that graph neural networks only perform low-pass filtering on feature vectors and do not have the non-linear manifold learning property. We further investigate their resilience to feature noise and propose some insights on GCN-based graph neural network design.