Is uniform expressivity too restrictive? Towards efficient expressivity of graph neural networks

TL;DR

This paper investigates the expressivity limits of graph neural networks, proving that many cannot achieve uniform expressivity with common activation functions, but can still efficiently express complex queries with limited parameters.

Contribution

It demonstrates the impossibility of uniform expressivity for GNNs with certain activations and proposes alternative efficient expressivity methods with logarithmic parameter growth.

Findings

Uniform expressivity is not achievable for GNNs with sigmoid and tanh activations.

Many GNNs can efficiently express GC2 queries with parameters logarithmic in graph degree.

A log-log dependency on graph degree is achievable with specific activation functions.

Abstract

Uniform expressivity guarantees that a Graph Neural Network (GNN) can express a query without the parameters depending on the size of the input graphs. This property is desirable in applications in order to have number of trainable parameters that is independent of the size of the input graphs. Uniform expressivity of the two variable guarded fragment (GC2) of first order logic is a well-celebrated result for Rectified Linear Unit (ReLU) GNNs [Barcelo & al., 2020]. In this article, we prove that uniform expressivity of GC2 queries is not possible for GNNs with a wide class of Pfaffian activation functions (including the sigmoid and tanh), answering a question formulated by [Grohe, 2021]. We also show that despite these limitations, many of those GNNs can still efficiently express GC2 queries in a way that the number of parameters remains logarithmic on the maximal degree of the input graphs. Furthermore, we demonstrate that a log-log dependency on the degree is achievable for a certain choice of activation function. This shows that uniform expressivity can be successfully relaxed by covering large graphs appearing in practical applications. Our experiments illustrates that our theoretical estimates hold in practice.