Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation

TL;DR

This paper introduces a retrieval-augmented training approach for robust named entity recognition (NER) that enhances noisy input representations by retrieving relevant text from a knowledge corpus, improving performance without needing gold text during inference.

Contribution

The paper proposes a novel retrieval-based method to improve noisy NER models using only noisy text and labels, with three retrieval strategies and a multi-view training framework.

Findings

Significant performance improvements in noisy NER tasks.

Effective retrieval methods enhance token representations.

Robust NER achieved without access to gold text during inference.

Abstract

Named entity recognition (NER) models often struggle with noisy inputs, such as those with spelling mistakes or errors generated by Optical Character Recognition processes, and learning a robust NER model is challenging. Existing robust NER models utilize both noisy text and its corresponding gold text for training, which is infeasible in many real-world applications in which gold text is not available. In this paper, we consider a more realistic setting in which only noisy text and its NER labels are available. We propose to retrieve relevant text of the noisy text from a knowledge corpus and use it to enhance the representation of the original noisy input. We design three retrieval methods: sparse retrieval based on lexicon similarity, dense retrieval based on semantic similarity, and self-retrieval based on task-specific text. After retrieving relevant text, we concatenate the retrieved text with the original noisy text and encode them with a transformer network, utilizing self-attention to enhance the contextual token representations of the noisy text using the retrieved text. We further employ a multi-view training framework that improves robust NER without retrieving text during inference. Experiments show that our retrieval-augmented model achieves significant improvements in various noisy NER settings.