Repetition Makes Perfect: Recurrent Graph Neural Networks Match Message-Passing Limit

TL;DR

This paper demonstrates that recurrent Graph Neural Networks with specific conditions can match the expressive power of the Weisfeiler-Leman algorithm, and with random initialization, can emulate all polynomial-time graph algorithms.

Contribution

It precisely characterizes the expressivity of recurrent GNNs, showing they match message-passing invariance limits and can emulate all polynomial-time algorithms with random initialization.

Findings

Recurrent GNNs can compute any graph algorithm respecting message-passing invariance.

Recurrent GNNs match the Weisfeiler-Leman power, unlike non-recurrent GNNs.

With random initialization, recurrent GNNs can emulate all polynomial-time graph algorithms.

Abstract

We precisely characterize the expressivity of computable Recurrent Graph Neural Networks (recurrent GNNs). We prove that recurrent GNNs with finite-precision parameters, sum aggregation, and ReLU activation, can compute any graph algorithm that respects the natural message-passing invariance induced by the Color Refinement (or Weisfeiler-Leman) algorithm. While it is well known that the expressive power of GNNs is limited by this invariance [Morris et al., AAAI 2019; Xu et al., ICLR 2019], we establish that recurrent GNNs can actually match this limit. This is in contrast to non-recurrent GNNs, which have the power of Weisfeiler-Leman only in a very weak, "non-uniform", sense where each graph size requires a different GNN to compute with. Our construction introduces only a polynomial overhead in both time and space. Furthermore, we show that by incorporating random initialization, for connected graphs recurrent GNNs can express all graph algorithms. In particular, any polynomial-time graph algorithm can be emulated on connected graphs in polynomial time by a recurrent GNN with random initialization.