Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay

TL;DR

This paper introduces ER-GNN, a framework for continual learning in graph neural networks that uses experience replay to mitigate catastrophic forgetting, with strategies for selecting representative experience nodes, validated on benchmark datasets.

Contribution

The paper proposes ER-GNN, a novel experience replay framework for continual graph learning, including three node selection strategies to improve knowledge retention in GNNs.

Findings

ER-GNN effectively reduces catastrophic forgetting in GNNs.

Experience node selection strategies improve learning efficiency.

Experimental results on benchmarks validate the approach.

Abstract

Graph Neural Networks (GNNs) have recently received significant research attention due to their superior performance on a variety of graph-related learning tasks. Most of the current works focus on either static or dynamic graph settings, addressing a single particular task, e.g., node/graph classification, link prediction. In this work, we investigate the question: can GNNs be applied to continuously learning a sequence of tasks? Towards that, we explore the Continual Graph Learning (CGL) paradigm and present the Experience Replay based framework ER-GNN for CGL to alleviate the catastrophic forgetting problem in existing GNNs. ER-GNN stores knowledge from previous tasks as experiences and replays them when learning new tasks to mitigate the catastrophic forgetting issue. We propose three experience node selection strategies: mean of feature, coverage maximization, and influence maximization, to guide the process of selecting experience nodes. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our ER-GNN and shed light on the incremental graph (non-Euclidean) structure learning.