Lifelong Learning of Graph Neural Networks for Open-World Node Classification

TL;DR

This paper introduces a lifelong learning framework for graph neural networks, addressing evolving graphs and new classes by analyzing implicit and explicit knowledge, and proposing an incremental training method evaluated on new datasets.

Contribution

It presents a novel incremental training approach for GNNs in lifelong learning scenarios, along with a new measure for historic data variance and extensive evaluation on multiple architectures and datasets.

Findings

50% receptive field retains 95% accuracy

Implicit knowledge gains importance with less explicit data

Effective lifelong learning on evolving graphs

Abstract

Graph neural networks (GNNs) have emerged as the standard method for numerous tasks on graph-structured data such as node classification. However, real-world graphs are often evolving over time and even new classes may arise. We model these challenges as an instance of lifelong learning, in which a learner faces a sequence of tasks and may take over knowledge acquired in past tasks. Such knowledge may be stored explicitly as historic data or implicitly within model parameters. In this work, we systematically analyze the influence of implicit and explicit knowledge. Therefore, we present an incremental training method for lifelong learning on graphs and introduce a new measure based on $k$-neighborhood time differences to address variances in the historic data. We apply our training method to five representative GNN architectures and evaluate them on three new lifelong node classification datasets. Our results show that no more than 50% of the GNN's receptive field is necessary to retain at least 95% accuracy compared to training over the complete history of the graph data. Furthermore, our experiments confirm that implicit knowledge becomes more important when fewer explicit knowledge is available.