Understanding the Message Passing in Graph Neural Networks via Power Iteration Clustering

TL;DR

This paper reveals that message passing in GNNs can be understood through power iteration, introducing SPIC models that improve processing of random networks and challenge existing GNN design assumptions.

Contribution

It provides a novel theoretical perspective linking GNN message passing to power iteration and introduces SPIC models that enhance GNN capabilities.

Findings

SPIC models extend GNNs and improve processing of random networks.

Redundancy in some state-of-the-art GNNs is demonstrated.

A lower limit for model evaluation with a random aggregator is defined.

Abstract

The mechanism of message passing in graph neural networks (GNNs) is still mysterious. Apart from convolutional neural networks, no theoretical origin for GNNs has been proposed. To our surprise, message passing can be best understood in terms of power iteration. By fully or partly removing activation functions and layer weights of GNNs, we propose subspace power iteration clustering (SPIC) models that iteratively learn with only one aggregator. Experiments show that our models extend GNNs and enhance their capability to process random featured networks. Moreover, we demonstrate the redundancy of some state-of-the-art GNNs in design and define a lower limit for model evaluation by a random aggregator of message passing. Our findings push the boundaries of the theoretical understanding of neural networks.