NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data

TL;DR

NuNER is a specialized NER model pre-trained using LLM-annotated data, achieving high performance in few-shot settings and demonstrating the importance of dataset diversity for effective entity recognition.

Contribution

The paper introduces NuNER, a compact, task-specific NER encoder pre-trained with LLM-generated annotations, enhancing data efficiency and performance over similar models.

Findings

NuNER outperforms similar-sized models in few-shot NER tasks.

Dataset size and entity diversity are crucial for NuNER's success.

NuNER competes with larger LLMs in NER performance.

Abstract

Large Language Models (LLMs) have shown impressive abilities in data annotation, opening the way for new approaches to solve classic NLP problems. In this paper, we show how to use LLMs to create NuNER, a compact language representation model specialized in the Named Entity Recognition (NER) task. NuNER can be fine-tuned to solve downstream NER problems in a data-efficient way, outperforming similar-sized foundation models in the few-shot regime and competing with much larger LLMs. We find that the size and entity-type diversity of the pre-training dataset are key to achieving good performance. We view NuNER as a member of the broader family of task-specific foundation models, recently unlocked by LLMs.