On the Effectiveness of Random Weights in Graph Neural Networks

TL;DR

This paper shows that using random weights in Graph Neural Networks can achieve performance comparable to trained models, significantly reducing training time and memory, and addressing feature rank collapse issues.

Contribution

It demonstrates the effectiveness of random weights in GNNs, providing a lightweight alternative that maintains predictive power and improves computational efficiency.

Findings

Random weights achieve comparable accuracy to trained weights.

Using random weights reduces training time by up to 6×.

Memory usage is decreased by up to 3× with random weights.

Abstract

Graph Neural Networks (GNNs) have achieved remarkable success across diverse tasks on graph-structured data, primarily through the use of learned weights in message passing layers. In this paper, we demonstrate that random weights can be surprisingly effective, achieving performance comparable to end-to-end training counterparts, across various tasks and datasets. Specifically, we show that by replacing learnable weights with random weights, GNNs can retain strong predictive power, while significantly reducing training time by up to 6$\times$ and memory usage by up to 3$\times$. Moreover, the random weights combined with our construction yield random graph propagation operators, which we show to reduce the problem of feature rank collapse in GNNs. These understandings and empirical results highlight random weights as a lightweight and efficient alternative, offering a compelling perspective on the design and training of GNN architectures.