Retrieval-Augmented Generation-based Relation Extraction

TL;DR

This paper introduces RAG4RE, a retrieval-augmented generation method that improves relation extraction accuracy by leveraging large language models and retrieval techniques, outperforming traditional methods on key benchmarks.

Contribution

The paper proposes RAG4RE, a novel retrieval-augmented approach that enhances relation extraction performance using LLMs and retrieval, addressing data and resource limitations of existing methods.

Findings

RAG4RE outperforms traditional LLM-based RE approaches on TACRED.

The approach shows significant improvements on TACRED and TACREV datasets.

RAG4RE demonstrates the potential of retrieval-augmented methods for relation extraction.

Abstract

Information Extraction (IE) is a transformative process that converts unstructured text data into a structured format by employing entity and relation extraction (RE) methodologies. The identification of the relation between a pair of entities plays a crucial role within this framework. Despite the existence of various techniques for relation extraction, their efficacy heavily relies on access to labeled data and substantial computational resources. In addressing these challenges, Large Language Models (LLMs) emerge as promising solutions; however, they might return hallucinating responses due to their own training data. To overcome these limitations, Retrieved-Augmented Generation-based Relation Extraction (RAG4RE) in this work is proposed, offering a pathway to enhance the performance of relation extraction tasks. This work evaluated the effectiveness of our RAG4RE approach utilizing different LLMs. Through the utilization of established benchmarks, such as TACRED, TACREV, Re-TACRED, and SemEval RE datasets, our aim is to comprehensively evaluate the efficacy of our RAG4RE approach. In particularly, we leverage prominent LLMs including Flan T5, Llama2, and Mistral in our investigation. The results of our study demonstrate that our RAG4RE approach surpasses performance of traditional RE approaches based solely on LLMs, particularly evident in the TACRED dataset and its variations. Furthermore, our approach exhibits remarkable performance compared to previous RE methodologies across both TACRED and TACREV datasets, underscoring its efficacy and potential for advancing RE tasks in natural language processing.