LinkNER: Linking Local Named Entity Recognition Models to Large Language Models using Uncertainty

TL;DR

LinkNER combines small fine-tuned NER models with large language models using an uncertainty-based linking strategy, improving robustness and performance on standard and noisy datasets.

Contribution

The paper introduces LinkNER, a novel framework that effectively links fine-tuned NER models with LLMs using uncertainty estimation, enhancing NER robustness and performance.

Findings

LinkNER surpasses state-of-the-art models in robustness tests.

Uncertainty estimation significantly improves model linking effectiveness.

Combining models enhances performance on noisy social media datasets.

Abstract

Named Entity Recognition (NER) serves as a fundamental task in natural language understanding, bearing direct implications for web content analysis, search engines, and information retrieval systems. Fine-tuned NER models exhibit satisfactory performance on standard NER benchmarks. However, due to limited fine-tuning data and lack of knowledge, it performs poorly on unseen entity recognition. As a result, the usability and reliability of NER models in web-related applications are compromised. Instead, Large Language Models (LLMs) like GPT-4 possess extensive external knowledge, but research indicates that they lack specialty for NER tasks. Furthermore, non-public and large-scale weights make tuning LLMs difficult. To address these challenges, we propose a framework that combines small fine-tuned models with LLMs (LinkNER) and an uncertainty-based linking strategy called RDC that enables fine-tuned models to complement black-box LLMs, achieving better performance. We experiment with both standard NER test sets and noisy social media datasets. LinkNER enhances NER task performance, notably surpassing SOTA models in robustness tests. We also quantitatively analyze the influence of key components like uncertainty estimation methods, LLMs, and in-context learning on diverse NER tasks, offering specific web-related recommendations. Code is available at https://github.com/zhzhengit/LinkNER.