LLMs are Better Than You Think: Label-Guided In-Context Learning for Named Entity Recognition

TL;DR

This paper introduces DEER, a label-guided in-context learning method for NER that uses label-grounded token statistics to improve demonstration relevance and model performance, rivaling supervised fine-tuning.

Contribution

DEER is a training-free ICL approach that leverages label-grounded token statistics for more effective demonstrations in NER tasks, outperforming existing ICL methods.

Findings

DEER outperforms existing ICL methods on five NER datasets.

DEER achieves results comparable to supervised fine-tuning.

DEER improves example retrieval and robustness in low-resource settings.

Abstract

In-context learning (ICL) enables large language models (LLMs) to perform new tasks using only a few demonstrations. However, in Named Entity Recognition (NER), existing ICL methods typically rely on task-agnostic semantic similarity for demonstration retrieval, which often yields less relevant examples and leads to inferior results. We introduce DEER, a training-free ICL approach that enables LLMs to make more informed entity predictions through the use of label-grounded statistics. DEER leverages token-level statistics from training labels to identify tokens most informative for entity recognition, enabling entity-focused demonstrations. It further uses these statistics to detect and refine error-prone tokens through a targeted reflection step. Evaluated on five NER datasets across four LLMs, DEER consistently outperforms existing ICL methods and achieves performance comparable to supervised fine-tuning. Further analyses demonstrate that DEER improves example retrieval, remains effective on both seen and unseen entities, and exhibits strong robustness in low-resource settings.