Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective

TL;DR

This paper introduces a novel prompt pool approach for continual relation extraction that captures within-task variance and consolidates knowledge without data storage, outperforming existing methods.

Contribution

It proposes a task-specific prompt pool and a generative model to better handle variances and mitigate forgetting in continual relation extraction.

Findings

Outperforms state-of-the-art prompt-based methods.

Eliminates need for explicit data storage.

Effectively captures within-task and cross-task variances.

Abstract

To address catastrophic forgetting in Continual Relation Extraction (CRE), many current approaches rely on memory buffers to rehearse previously learned knowledge while acquiring new tasks. Recently, prompt-based methods have emerged as potent alternatives to rehearsal-based strategies, demonstrating strong empirical performance. However, upon analyzing existing prompt-based approaches for CRE, we identified several critical limitations, such as inaccurate prompt selection, inadequate mechanisms for mitigating forgetting in shared parameters, and suboptimal handling of cross-task and within-task variances. To overcome these challenges, we draw inspiration from the relationship between prefix-tuning and mixture of experts, proposing a novel approach that employs a prompt pool for each task, capturing variations within each task while enhancing cross-task variances. Furthermore, we incorporate a generative model to consolidate prior knowledge within shared parameters, eliminating the need for explicit data storage. Extensive experiments validate the efficacy of our approach, demonstrating superior performance over state-of-the-art prompt-based and rehearsal-free methods in continual relation extraction.