Retrieval Augmented Instruction Tuning for Open NER with Large Language Models

TL;DR

This paper introduces Retrieval Augmented Instruction Tuning (RA-IT), a method that enhances open NER performance in large language models by retrieving similar training examples to provide context, validated across English and Chinese datasets.

Contribution

The paper proposes RA-IT, a novel approach combining retrieval and instruction tuning for open NER, and provides a new Chinese IT dataset for comprehensive evaluation.

Findings

RA-IT improves NER performance across data sizes and languages.

Retrieval strategies significantly impact the effectiveness of RA-IT.

Experimental results confirm the robustness of RA-IT in English and Chinese scenarios.

Abstract

The strong capability of large language models (LLMs) has been applied to information extraction (IE) through either retrieval augmented prompting or instruction tuning (IT). However, the best way to incorporate information with LLMs for IE remains an open question. In this paper, we explore Retrieval Augmented Instruction Tuning (RA-IT) for IE, focusing on the task of open named entity recognition (NER). Specifically, for each training sample, we retrieve semantically similar examples from the training dataset as the context and prepend them to the input of the original instruction. To evaluate our RA-IT approach more thoroughly, we construct a Chinese IT dataset for open NER and evaluate RA-IT in both English and Chinese scenarios. Experimental results verify the effectiveness of RA-IT across various data sizes and in both English and Chinese scenarios. We also conduct thorough studies to explore the impacts of various retrieval strategies in the proposed RA-IT framework. Code and data are available at: https://github.com/Emma1066/Retrieval-Augmented-IT-OpenNER