Demystifying MPNNs: Message Passing as Merely Efficient Matrix Multiplication

TL;DR

This paper provides a theoretical framework for understanding GNNs, showing that message passing is essentially efficient matrix multiplication, and analyzes factors affecting GNN performance and behavior.

Contribution

It offers a comprehensive theoretical analysis of GNNs, clarifying how message passing relates to matrix operations and how various factors influence GNN effectiveness.

Findings

k-layer GNNs efficiently aggregate k-hop neighborhoods

Loop structures significantly influence neighborhood computation

Normalization schemes impact GNN performance

Abstract

While Graph Neural Networks (GNNs) have achieved remarkable success, their design largely relies on empirical intuition rather than theoretical understanding. In this paper, we present a comprehensive analysis of GNN behavior through three fundamental aspects: (1) we establish that \textbf{$k$-layer} Message Passing Neural Networks efficiently aggregate \textbf{$k$-hop} neighborhood information through iterative computation, (2) analyze how different loop structures influence neighborhood computation, and (3) examine behavior across structure-feature hybrid and structure-only tasks. For deeper GNNs, we demonstrate that gradient-related issues, rather than just over-smoothing, can significantly impact performance in sparse graphs. We also analyze how different normalization schemes affect model performance and how GNNs make predictions with uniform node features, providing a theoretical framework that bridges the gap between empirical success and theoretical understanding.