A Generative Adaptive Replay Continual Learning Model for Temporal Knowledge Graph Reasoning

TL;DR

This paper introduces DGAR, a novel continual learning approach for temporal knowledge graph reasoning that generates and adaptively replays historical context representations, improving reasoning accuracy and reducing forgetting.

Contribution

The paper proposes a Deep Generative Adaptive Replay method that leverages a diffusion model and context prompts to better preserve and utilize historical information in TKGR.

Findings

DGAR outperforms baseline methods in reasoning accuracy.

DGAR effectively mitigates catastrophic forgetting.

The adaptive replay mechanism enhances integration of historical and current data.

Abstract

Recent Continual Learning (CL)-based Temporal Knowledge Graph Reasoning (TKGR) methods focus on significantly reducing computational cost and mitigating catastrophic forgetting caused by fine-tuning models with new data. However, existing CL-based TKGR methods still face two key limitations: (1) They usually one-sidedly reorganize individual historical facts, while overlooking the historical context essential for accurately understanding the historical semantics of these facts; (2) They preserve historical knowledge by simply replaying historical facts, while ignoring the potential conflicts between historical and emerging facts. In this paper, we propose a Deep Generative Adaptive Replay (DGAR) method, which can generate and adaptively replay historical entity distribution representations from the whole historical context. To address the first challenge, historical context prompts as sampling units are built to preserve the whole historical context information. To overcome the second challenge, a pre-trained diffusion model is adopted to generate the historical distribution. During the generation process, the common features between the historical and current distributions are enhanced under the guidance of the TKGR model. In addition, a layer-by-layer adaptive replay mechanism is designed to effectively integrate historical and current distributions. Experimental results demonstrate that DGAR significantly outperforms baselines in reasoning and mitigating forgetting.