On the expressive power of message-passing neural networks as global feature map transformers

TL;DR

This paper examines the expressive capabilities of message-passing neural networks (MPNNs) in transforming graph node features, introducing the concept of global feature map transformers (GFMT) and analyzing their relation to a formal language MPLang.

Contribution

It formalizes the notion of GFMT, compares MPNNs' expressiveness to MPLang, and explores exact and approximate transformations under various activation function constraints.

Findings

MPNNs can be represented within MPLang.

The paper clarifies the extent of MPNNs' expressive power.

Analysis includes ReLU and arbitrary activation functions.

Abstract

We investigate the power of message-passing neural networks (MPNNs) in their capacity to transform the numerical features stored in the nodes of their input graphs. Our focus is on global expressive power, uniformly over all input graphs, or over graphs of bounded degree with features from a bounded domain. Accordingly, we introduce the notion of a global feature map transformer (GFMT). As a yardstick for expressiveness, we use a basic language for GFMTs, which we call MPLang. Every MPNN can be expressed in MPLang, and our results clarify to which extent the converse inclusion holds. We consider exact versus approximate expressiveness; the use of arbitrary activation functions; and the case where only the ReLU activation function is allowed.