A Multi-Agent LLM Framework for Multi-Domain Low-Resource In-Context NER via Knowledge Retrieval, Disambiguation and Reflective Analysis

TL;DR

This paper introduces KDR-Agent, a multi-agent framework that enhances low-resource, multi-domain NER by integrating knowledge retrieval, disambiguation, and reflection, significantly improving zero-shot and few-shot performance across diverse datasets.

Contribution

The paper presents a novel multi-agent framework that combines knowledge retrieval, disambiguation, and reflective analysis to improve in-context NER in low-resource, multi-domain settings.

Findings

Outperforms existing zero-shot and few-shot ICL baselines.

Effectively leverages Wikipedia for factual knowledge retrieval.

Demonstrates robustness across ten datasets from five domains.

Abstract

In-context learning (ICL) with large language models (LLMs) has emerged as a promising paradigm for named entity recognition (NER) in low-resource scenarios. However, existing ICL-based NER methods suffer from three key limitations: (1) reliance on dynamic retrieval of annotated examples, which is problematic when annotated data is scarce; (2) limited generalization to unseen domains due to the LLM's insufficient internal domain knowledge; and (3) failure to incorporate external knowledge or resolve entity ambiguities. To address these challenges, we propose KDR-Agent, a novel multi-agent framework for multi-domain low-resource in-context NER that integrates Knowledge retrieval, Disambiguation, and Reflective analysis. KDR-Agent leverages natural-language type definitions and a static set of entity-level contrastive demonstrations to reduce dependency on large annotated corpora. A central planner coordinates specialized agents to (i) retrieve factual knowledge from Wikipedia for domain-specific mentions, (ii) resolve ambiguous entities via contextualized reasoning, and (iii) reflect on and correct model predictions through structured self-assessment. Experiments across ten datasets from five domains demonstrate that KDR-Agent significantly outperforms existing zero-shot and few-shot ICL baselines across multiple LLM backbones. The code and data can be found at https://github.com/MWXGOD/KDR-Agent.