What graph neural networks cannot learn: depth vs width

TL;DR

This paper explores the expressive limits of message-passing graph neural networks, showing they are Turing universal with sufficient depth and width, but can lose power when these are restricted, with implications for graph-based tasks.

Contribution

It establishes the conditions under which GNNs are Turing universal and demonstrates the limitations imposed by restricted depth and width, using novel techniques from distributed computing.

Findings

GNNs are Turing universal with enough depth and width.

Restricted depth and width can significantly impair GNNs' expressive power.

Certain graph problems are impossible for GNNs unless their size exceeds polynomial bounds.

Abstract

This paper studies the expressive power of graph neural networks falling within the message-passing framework (GNNmp). Two results are presented. First, GNNmp are shown to be Turing universal under sufficient conditions on their depth, width, node attributes, and layer expressiveness. Second, it is discovered that GNNmp can lose a significant portion of their power when their depth and width is restricted. The proposed impossibility statements stem from a new technique that enables the repurposing of seminal results from distributed computing and leads to lower bounds for an array of decision, optimization, and estimation problems involving graphs. Strikingly, several of these problems are deemed impossible unless the product of a GNNmp's depth and width exceeds a polynomial of the graph size; this dependence remains significant even for tasks that appear simple or when considering approximation.