# A Neural Span-Based Continual Named Entity Recognition Model

## TL;DR

This paper introduces SpanKL, a span-based continual NER model utilizing knowledge distillation and multi-label prediction, effectively mitigating forgetting and outperforming previous methods on synthetic datasets.

## Contribution

The paper presents a novel span-based continual NER model, SpanKL, with techniques to preserve memory and prevent conflicts, advancing the state-of-the-art in incremental learning for NER.

## Key findings

- SpanKL outperforms previous SOTA methods on synthetic datasets.
- SpanKL achieves a small gap from the upper bound, indicating high practical value.
- The model effectively mitigates forgetting in continual NER tasks.

## Abstract

Named Entity Recognition (NER) models capable of Continual Learning (CL) are realistically valuable in areas where entity types continuously increase (e.g., personal assistants). Meanwhile the learning paradigm of NER advances to new patterns such as the span-based methods. However, its potential to CL has not been fully explored. In this paper, we propose SpanKL, a simple yet effective Span-based model with Knowledge distillation (KD) to preserve memories and multi-Label prediction to prevent conflicts in CL-NER. Unlike prior sequence labeling approaches, the inherently independent modeling in span and entity level with the designed coherent optimization on SpanKL promotes its learning at each incremental step and mitigates the forgetting. Experiments on synthetic CL datasets derived from OntoNotes and Few-NERD show that SpanKL significantly outperforms previous SoTA in many aspects, and obtains the smallest gap from CL to the upper bound revealing its high practiced value. The code is available at https://github.com/Qznan/SpanKL.