Partially Supervised Named Entity Recognition via the Expected Entity Ratio Loss

TL;DR

This paper introduces the Expected Entity Ratio loss for learning named entity recognizers with missing annotations, demonstrating improved performance across multiple languages and annotation scenarios.

Contribution

It proposes a novel loss function for partially supervised NER that is both theoretically justified and empirically effective, outperforming previous methods.

Findings

Outperforms state-of-the-art baselines by up to +12.7 F1 score in low-annotation settings.

Effective across various languages and annotation scenarios.

Sparse annotation combined with the new loss surpasses exhaustive annotation for limited budgets.

Abstract

We study learning named entity recognizers in the presence of missing entity annotations. We approach this setting as tagging with latent variables and propose a novel loss, the Expected Entity Ratio, to learn models in the presence of systematically missing tags. We show that our approach is both theoretically sound and empirically useful. Experimentally, we find that it meets or exceeds performance of strong and state-of-the-art baselines across a variety of languages, annotation scenarios, and amounts of labeled data. In particular, we find that it significantly outperforms the previous state-of-the-art methods from Mayhew et al. (2019) and Li et al. (2021) by +12.7 and +2.3 F1 score in a challenging setting with only 1,000 biased annotations, averaged across 7 datasets. We also show that, when combined with our approach, a novel sparse annotation scheme outperforms exhaustive annotation for modest annotation budgets.