Learning to Execute Graph Algorithms Exactly with Graph Neural Networks

TL;DR

This paper proves that graph neural networks can exactly learn to execute certain graph algorithms by training MLP ensembles on local instructions, enabling error-free inference for algorithms like BFS and Bellman-Ford.

Contribution

It introduces a theoretical framework showing GNNs can learn to execute graph algorithms exactly under bounded-degree and finite-precision constraints, using NTK theory.

Findings

GNNs can learn local instructions with small training sets

Exact algorithm execution during inference is achievable with high probability

Positive results for algorithms like BFS, DFS, and Bellman-Ford

Abstract

Understanding what graph neural networks can learn, especially their ability to learn to execute algorithms, remains a central theoretical challenge. In this work, we prove exact learnability results for graph algorithms under bounded-degree and finite-precision constraints. Our approach follows a two-step process. First, we train an ensemble of multi-layer perceptrons (MLPs) to execute the local instructions of a single node. Second, during inference, we use the trained MLP ensemble as the update function within a graph neural network (GNN). Leveraging Neural Tangent Kernel (NTK) theory, we show that local instructions can be learned from a small training set, enabling the complete graph algorithm to be executed during inference without error and with high probability. To illustrate the learning power of our setting, we establish a rigorous learnability result for the LOCAL model of distributed computation. We further demonstrate positive learnability results for widely studied algorithms such as message flooding, breadth-first and depth-first search, and Bellman-Ford.