On the Universality of Graph Neural Networks on Large Random Graphs

TL;DR

This paper investigates the approximation capabilities of Graph Neural Networks (GNNs) on large random graphs, demonstrating that augmenting GNNs with unique node identifiers enhances their power and universality across various models.

Contribution

It introduces the concept of Structural GNNs (SGNNs) with node identifiers, proving their universality and superior approximation power over traditional GNNs on large random graph models.

Findings

SGNNs outperform GNNs in distinguishing communities in SBMs.

c-SGNNs are more powerful than c-GNNs in the continuous limit.

Universality of SGNNs on multiple random graph models is established.

Abstract

We study the approximation power of Graph Neural Networks (GNNs) on latent position random graphs. In the large graph limit, GNNs are known to converge to certain "continuous" models known as c-GNNs, which directly enables a study of their approximation power on random graph models. In the absence of input node features however, just as GNNs are limited by the Weisfeiler-Lehman isomorphism test, c-GNNs will be severely limited on simple random graph models. For instance, they will fail to distinguish the communities of a well-separated Stochastic Block Model (SBM) with constant degree function. Thus, we consider recently proposed architectures that augment GNNs with unique node identifiers, referred to as Structural GNNs here (SGNNs). We study the convergence of SGNNs to their continuous counterpart (c-SGNNs) in the large random graph limit, under new conditions on the node identifiers. We then show that c-SGNNs are strictly more powerful than c-GNNs in the continuous limit, and prove their universality on several random graph models of interest, including most SBMs and a large class of random geometric graphs. Our results cover both permutation-invariant and permutation-equivariant architectures.