What Functions Can Graph Neural Networks Generate?

TL;DR

This paper characterizes the functions that Graph Neural Networks (GNNs) can generate through permutation compatibility, providing a formal framework that links GNN capabilities to algebraic conditions and enabling the generation of various graph functions.

Contribution

It introduces permutation compatibility as a key algebraic condition, proving GNNs can generate any permutation compatible function with feature augmentation, and offers a basis function representation theorem.

Findings

GNNs are necessarily permutation compatible functions.

Any permutation compatible function can be generated by a GNN with feature augmentation.

GNNs can generate key graph problems like max-flow and min-cut with simple feature augmentation.

Abstract

In this paper, we fully answer the above question through a key algebraic condition on graph functions, called \textit{permutation compatibility}, that relates permutations of weights and features of the graph to functional constraints. We prove that: (i) a GNN, as a graph function, is necessarily permutation compatible; (ii) conversely, any permutation compatible function, when restricted on input graphs with distinct node features, can be generated by a GNN; (iii) for arbitrary node features (not necessarily distinct), a simple feature augmentation scheme suffices to generate a permutation compatible function by a GNN; (iv) permutation compatibility can be verified by checking only quadratically many functional constraints, rather than an exhaustive search over all the permutations; (v) GNNs can generate \textit{any} graph function once we augment the node features with node identities, thus going beyond graph isomorphism and permutation compatibility. The above characterizations pave the path to formally study the intricate connection between GNNs and other algorithmic procedures on graphs. For instance, our characterization implies that many natural graph problems, such as min-cut value, max-flow value, max-clique size, and shortest path can be generated by a GNN using a simple feature augmentation. In contrast, the celebrated Weisfeiler-Lehman graph-isomorphism test fails whenever a permutation compatible function with identical features cannot be generated by a GNN. At the heart of our analysis lies a novel representation theorem that identifies basis functions for GNNs. This enables us to translate the properties of the target graph function into properties of the GNN's aggregation function.