Prompt-Driven Continual Graph Learning

TL;DR

This paper introduces PROMPTCGL, a prompt-based continual graph learning framework that maintains fixed graph neural networks, avoiding catastrophic forgetting and reducing memory use through hierarchical prompting and personalized prompt generation.

Contribution

It proposes a novel prompt-driven approach for continual graph learning that addresses scalability and privacy issues, with hierarchical prompts and a personalized prompt generator.

Findings

Outperforms existing CGL methods on four benchmarks.

Reduces memory consumption significantly.

Effectively prevents catastrophic forgetting.

Abstract

Continual Graph Learning (CGL), which aims to accommodate new tasks over evolving graph data without forgetting prior knowledge, is garnering significant research interest. Mainstream solutions adopt the memory replay-based idea, ie, caching representative data from earlier tasks for retraining the graph model. However, this strategy struggles with scalability issues for constantly evolving graphs and raises concerns regarding data privacy. Inspired by recent advancements in the prompt-based learning paradigm, this paper introduces a novel prompt-driven continual graph learning (PROMPTCGL) framework, which learns a separate prompt for each incoming task and maintains the underlying graph neural network model fixed. In this way, PROMPTCGL naturally avoids catastrophic forgetting of knowledge from previous tasks. More specifically, we propose hierarchical prompting to instruct the model from both feature- and topology-level to fully address the variability of task graphs in dynamic continual learning. Additionally, we develop a personalized prompt generator to generate tailored prompts for each graph node while minimizing the number of prompts needed, leading to constant memory consumption regardless of the graph scale. Extensive experiments on four benchmarks show that PROMPTCGL achieves superior performance against existing CGL approaches while significantly reducing memory consumption. Our code is available at https://github.com/QiWang98/PromptCGL.