Rationale-Enhanced Language Models are Better Continual Relation Learners

TL;DR

This paper introduces a rationale-enhanced approach for continual relation extraction, leveraging explanations from large language models to improve robustness and reduce catastrophic forgetting, leading to superior performance on standard benchmarks.

Contribution

It proposes a multi-task rationale tuning strategy and contrastive rationale replay to enhance continual relation learning with explanations from LLMs.

Findings

Outperforms state-of-the-art CRE models on benchmarks

Rationale tuning improves relation robustness

Contrastive rationale replay distinguishes analogous relations

Abstract

Continual relation extraction (CRE) aims to solve the problem of catastrophic forgetting when learning a sequence of newly emerging relations. Recent CRE studies have found that catastrophic forgetting arises from the model's lack of robustness against future analogous relations. To address the issue, we introduce rationale, i.e., the explanations of relation classification results generated by large language models (LLM), into CRE task. Specifically, we design the multi-task rationale tuning strategy to help the model learn current relations robustly. We also conduct contrastive rationale replay to further distinguish analogous relations. Experimental results on two standard benchmarks demonstrate that our method outperforms the state-of-the-art CRE models.