Towards Continuous Reuse of Graph Models via Holistic Memory Diversification

TL;DR

This paper proposes a holistic memory diversification framework for incremental graph learning, enhancing memory selection and generation to improve continual learning performance on complex graph tasks.

Contribution

It introduces a novel diversified memory selection and generation approach that considers intra- and inter-class diversities for better memory replay in graph models.

Findings

Outperforms state-of-the-art methods on multiple datasets.

Effective memory diversification improves continual learning.

Memory generation enhances knowledge retention in graphs.

Abstract

This paper addresses the challenge of incremental learning in growing graphs with increasingly complex tasks. The goal is to continuously train a graph model to handle new tasks while retaining proficiency in previous tasks via memory replay. Existing methods usually overlook the importance of memory diversity, limiting in selecting high-quality memory from previous tasks and remembering broad previous knowledge within the scarce memory on graphs. To address that, we introduce a novel holistic Diversified Memory Selection and Generation (DMSG) framework for incremental learning in graphs, which first introduces a buffer selection strategy that considers both intra-class and inter-class diversities, employing an efficient greedy algorithm for sampling representative training nodes from graphs into memory buffers after learning each new task. Then, to adequately rememorize the knowledge preserved in the memory buffer when learning new tasks, a diversified memory generation replay method is introduced. This method utilizes a variational layer to generate the distribution of buffer node embeddings and sample synthesized ones for replaying. Furthermore, an adversarial variational embedding learning method and a reconstruction-based decoder are proposed to maintain the integrity and consolidate the generalization of the synthesized node embeddings, respectively. Extensive experimental results on publicly accessible datasets demonstrate the superiority of \method{} over state-of-the-art methods.