WAVE++: Capturing Within-Task Variance for Continual Relation Extraction with Adaptive Prompting

TL;DR

WAVE++ introduces a prompt-based continual relation extraction method that captures within-task and cross-task variations without storing past data, using task-specific prompt pools, label descriptions, and a generative model to improve accuracy and privacy.

Contribution

It proposes a novel prompt-based CRE approach with task-specific prompt pools, label descriptions, and a generative model to better handle task variations and eliminate data storage.

Findings

Outperforms state-of-the-art methods in CRE tasks.

Effectively captures within-task and cross-task variations.

Reduces memory usage and privacy concerns.

Abstract

Memory-based approaches have shown strong performance in Continual Relation Extraction (CRE). However, storing examples from previous tasks increases memory usage and raises privacy concerns. Recently, prompt-based methods have emerged as a promising alternative, as they do not rely on storing past samples. Despite this progress, current prompt-based techniques face several core challenges in CRE, particularly in accurately identifying task identities and mitigating catastrophic forgetting. Existing prompt selection strategies often suffer from inaccuracies, lack robust mechanisms to prevent forgetting in shared parameters, and struggle to handle both cross-task and within-task variations. In this paper, we propose WAVE++, a novel approach inspired by the connection between prefix-tuning and mixture of experts. Specifically, we introduce task-specific prompt pools that enhance flexibility and adaptability across diverse tasks while avoiding boundary-spanning risks; this design more effectively captures both within-task and cross-task variations. To further refine relation classification, we incorporate label descriptions that provide richer, more global context, enabling the model to better distinguish among different relations. We also propose a training-free mechanism to improve task prediction during inference. Moreover, we integrate a generative model to consolidate prior knowledge within the shared parameters, thereby removing the need for explicit data storage. Extensive experiments demonstrate that WAVE++ outperforms state-of-the-art prompt-based and rehearsal-based methods, offering a more robust solution for continual relation extraction. Our code is publicly available at https://github.com/PiDinosauR2804/WAVE-CRE-PLUS-PLUS.