The Surprising Power of Graph Neural Networks with Random Node Initialization

TL;DR

This paper demonstrates that graph neural networks with random node initialization are universal in expressive power, surpassing traditional GNNs, and empirically shows their superior performance on specific datasets.

Contribution

It proves the universality of GNNs with random node initialization, a novel theoretical result, and empirically validates their improved performance over standard GNNs.

Findings

GNNs with RNI are universal in expressive power.

Empirical results show RNI-enhanced GNNs outperform standard GNNs.

RNI preserves invariance properties in expectation.

Abstract

Graph neural networks (GNNs) are effective models for representation learning on relational data. However, standard GNNs are limited in their expressive power, as they cannot distinguish graphs beyond the capability of the Weisfeiler-Leman graph isomorphism heuristic. In order to break this expressiveness barrier, GNNs have been enhanced with random node initialization (RNI), where the idea is to train and run the models with randomized initial node features. In this work, we analyze the expressive power of GNNs with RNI, and prove that these models are universal, a first such result for GNNs not relying on computationally demanding higher-order properties. This universality result holds even with partially randomized initial node features, and preserves the invariance properties of GNNs in expectation. We then empirically analyze the effect of RNI on GNNs, based on carefully constructed datasets. Our empirical findings support the superior performance of GNNs with RNI over standard GNNs.