Walking Out of the Weisfeiler Leman Hierarchy: Graph Learning Beyond Message Passing

TL;DR

CRaWl introduces a novel graph neural network architecture that leverages subgraph information along random walks, enabling detection of long-range interactions and surpassing traditional message passing GNNs in expressiveness.

Contribution

CRaWl provides a new approach to graph learning that is theoretically incomparable with GNNs and extends beyond the Weisfeiler Leman hierarchy, with empirical performance matching state-of-the-art methods.

Findings

CRaWl is more expressive than standard GNNs.

CRaWl achieves state-of-the-art results on benchmark datasets.

It effectively captures long-range and non-local graph features.

Abstract

We propose CRaWl, a novel neural network architecture for graph learning. Like graph neural networks, CRaWl layers update node features on a graph and thus can freely be combined or interleaved with GNN layers. Yet CRaWl operates fundamentally different from message passing graph neural networks. CRaWl layers extract and aggregate information on subgraphs appearing along random walks through a graph using 1D Convolutions. Thereby it detects long range interactions and computes non-local features. As the theoretical basis for our approach, we prove a theorem stating that the expressiveness of CRaWl is incomparable with that of the Weisfeiler Leman algorithm and hence with graph neural networks. That is, there are functions expressible by CRaWl, but not by GNNs and vice versa. This result extends to higher levels of the Weisfeiler Leman hierarchy and thus to higher-order GNNs. Empirically, we show that CRaWl matches state-of-the-art GNN architectures across a multitude of benchmark datasets for classification and regression on graphs.