Caterpillar GNN: Replacing Message Passing with Efficient Aggregation

TL;DR

This paper introduces Caterpillar GNNs, a new class of graph neural networks that replace traditional message passing with efficient walk-based aggregation, balancing expressivity and inductive bias for scalable and effective graph learning.

Contribution

It proposes a novel aggregation method over walk incidence matrices, providing a spectrum between message passing and simpler walk-based methods, with theoretical expressivity analysis and practical improvements.

Findings

Caterpillar GNNs match MPGNNs on benchmarks with fewer nodes in hidden layers.

The approach offers a trade-off between expressivity and inductive bias.

The method scales seamlessly between classical message passing and simpler walk-based techniques.

Abstract

Message-passing graph neural networks (MPGNNs) dominate modern graph learning. Typical efforts enhance MPGNN's expressive power by enriching the adjacency-based aggregation. In contrast, we introduce an efficient aggregation over walk incidence-based matrices that are constructed to deliberately trade off some expressivity for stronger and more structured inductive bias. Our approach allows for seamless scaling between classical message-passing and simpler methods based on walks. We rigorously characterize the expressive power at each intermediate step using homomorphism counts over a hierarchy of generalized caterpillar graphs. Based on this foundation, we propose Caterpillar GNNs, whose robust graph-level aggregation successfully tackles a benchmark specifically designed to challenge MPGNNs. Moreover, we demonstrate that, on real-world datasets, Caterpillar GNNs achieve comparable predictive performance while significantly reducing the number of nodes in the hidden layers of the computational graph.