Recognizing Nested Entities from Flat Supervision: A New NER Subtask, Feasibility and Challenges

TL;DR

This paper introduces a new nested-from-flat NER subtask, demonstrating that models trained on flat annotations can recognize nested entities effectively, highlighting feasibility and challenges in this approach.

Contribution

It proposes a span-based model for nested-from-flat NER, creates a new benchmark dataset, and analyzes the challenges due to data and annotation inconsistencies.

Findings

Model achieves over 54% F1 on nested spans in ACE datasets.

Performance on flat entities remains unaffected.

Main challenges are data and annotation inconsistencies.

Abstract

Many recent named entity recognition (NER) studies criticize flat NER for its non-overlapping assumption, and switch to investigating nested NER. However, existing nested NER models heavily rely on training data annotated with nested entities, while labeling such data is costly. This study proposes a new subtask, nested-from-flat NER, which corresponds to a realistic application scenario: given data annotated with flat entities only, one may still desire the trained model capable of recognizing nested entities. To address this task, we train span-based models and deliberately ignore the spans nested inside labeled entities, since these spans are possibly unlabeled entities. With nested entities removed from the training data, our model achieves 54.8%, 54.2% and 41.1% F1 scores on the subset of spans within entities on ACE 2004, ACE 2005 and GENIA, respectively. This suggests the effectiveness of our approach and the feasibility of the task. In addition, the model's performance on flat entities is entirely unaffected. We further manually annotate the nested entities in the test set of CoNLL 2003, creating a nested-from-flat NER benchmark. Analysis results show that the main challenges stem from the data and annotation inconsistencies between the flat and nested entities.