Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training

TL;DR

This paper introduces a noise-robust learning approach combined with language model-based self-training to improve distantly-supervised NER, effectively handling noisy labels and enhancing model performance on benchmark datasets.

Contribution

It proposes a novel noise-robust loss function and a self-training method with language model augmentations for distantly-supervised NER.

Findings

Significant performance improvements over existing methods.

Effective noise removal in distantly-labeled data.

Enhanced generalization through language model augmentations.

Abstract

We study the problem of training named entity recognition (NER) models using only distantly-labeled data, which can be automatically obtained by matching entity mentions in the raw text with entity types in a knowledge base. The biggest challenge of distantly-supervised NER is that the distant supervision may induce incomplete and noisy labels, rendering the straightforward application of supervised learning ineffective. In this paper, we propose (1) a noise-robust learning scheme comprised of a new loss function and a noisy label removal step, for training NER models on distantly-labeled data, and (2) a self-training method that uses contextualized augmentations created by pre-trained language models to improve the generalization ability of the NER model. On three benchmark datasets, our method achieves superior performance, outperforming existing distantly-supervised NER models by significant margins.