Can Graph Neural Networks Go "Online"? An Analysis of Pretraining and Inference

TL;DR

This paper investigates whether pretraining or retraining is more effective for graph neural networks when new nodes and edges appear in dynamic graph data, showing that pretraining generally performs better.

Contribution

It provides an experimental comparison between adapting pretrained GNNs and retraining from scratch in online scenarios, highlighting the advantages of pretraining.

Findings

Pretrained GNNs achieve high accuracy on unseen nodes.

Pretraining is preferable over retraining from scratch in dynamic settings.

First experimental step towards online GNN variants.

Abstract

Large-scale graph data in real-world applications is often not static but dynamic, i. e., new nodes and edges appear over time. Current graph convolution approaches are promising, especially, when all the graph's nodes and edges are available during training. When unseen nodes and edges are inserted after training, it is not yet evaluated whether up-training or re-training from scratch is preferable. We construct an experimental setup, in which we insert previously unseen nodes and edges after training and conduct a limited amount of inference epochs. In this setup, we compare adapting pretrained graph neural networks against retraining from scratch. Our results show that pretrained models yield high accuracy scores on the unseen nodes and that pretraining is preferable over retraining from scratch. Our experiments represent a first step to evaluate and develop truly online variants of graph neural networks.