Improving Continual Relation Extraction by Distinguishing Analogous Semantics

TL;DR

This paper presents a novel continual relation extraction model that effectively distinguishes analogous relations and mitigates overfitting by using memory-insensitive prototypes, memory augmentation, and specialized training techniques.

Contribution

The paper introduces a new model with memory-insensitive prototypes and augmentation, improving continual relation extraction performance on analogous relations and reducing overfitting.

Findings

Model outperforms existing methods in distinguishing analogous relations.

Memory augmentation enhances the model's ability to retain learned relations.

Proposed techniques effectively mitigate overfitting in continual learning.

Abstract

Continual relation extraction (RE) aims to learn constantly emerging relations while avoiding forgetting the learned relations. Existing works store a small number of typical samples to re-train the model for alleviating forgetting. However, repeatedly replaying these samples may cause the overfitting problem. We conduct an empirical study on existing works and observe that their performance is severely affected by analogous relations. To address this issue, we propose a novel continual extraction model for analogous relations. Specifically, we design memory-insensitive relation prototypes and memory augmentation to overcome the overfitting problem. We also introduce integrated training and focal knowledge distillation to enhance the performance on analogous relations. Experimental results show the superiority of our model and demonstrate its effectiveness in distinguishing analogous relations and overcoming overfitting.