Recurrent Graph Neural Networks and Arithmetic Circuits

TL;DR

This paper establishes a precise equivalence between the computational power of recurrent graph neural networks and recurrent arithmetic circuits over real numbers, linking neural network expressivity with circuit complexity theory.

Contribution

It introduces recurrent arithmetic circuits as a new model and proves their computational equivalence with recurrent GNNs, expanding understanding of neural network capabilities.

Findings

Recurrent GNNs can simulate recurrent arithmetic circuits.

Recurrent arithmetic circuits encode labelled graphs as real-valued tuples.

The paper deepens the theoretical understanding of neural network expressivity.

Abstract

We characterise the computational power of recurrent graph neural networks (GNNs) in terms of arithmetic circuits over the real numbers. Our networks are not restricted to aggregate-combine GNNs or other particular types. Generalising similar notions from the literature, we introduce the model of recurrent arithmetic circuits, which can be seen as arithmetic analogues of sequential or logical circuits. These circuits utilise so-called memory gates which are used to store data between iterations of the recurrent circuit. While (recurrent) GNNs work on labelled graphs, we construct arithmetic circuits that obtain encoded labelled graphs as real valued tuples and then compute the same function. For the other direction we construct recurrent GNNs which are able to simulate the computations of recurrent circuits. These GNNs are given the circuit-input as initial feature vectors and then, after the GNN-computation, have the circuit-output among the feature vectors of its nodes. In this way we establish an exact correspondence between the expressivity of recurrent GNNs and recurrent arithmetic circuits operating over real numbers. Our results both deepen our understanding of the capabilities of trained neural networks and open new approaches to study recurrent neural networks using the lens of circuit complexity theory.