End-to-End Trainable Retrieval-Augmented Generation for Relation Extraction

TL;DR

This paper introduces ETRAG, an end-to-end trainable retrieval-augmented generation model for relation extraction, enabling the retriever to be optimized directly for the task, leading to improved performance on TACRED.

Contribution

The paper presents a novel differentiable retrieval method allowing end-to-end training of retrieval-augmented generation models for relation extraction.

Findings

ETRAG outperforms baseline models on TACRED.

Retrieved instances contain relevant relation labels and entities.

End-to-end training improves retrieval relevance for relation extraction.

Abstract

This paper addresses a crucial challenge in retrieval-augmented generation-based relation extractors; the end-to-end training is not applicable to conventional retrieval-augmented generation due to the non-differentiable nature of instance retrieval. This problem prevents the instance retrievers from being optimized for the relation extraction task, and conventionally it must be trained with an objective different from that for relation extraction. To address this issue, we propose a novel End-to-end Trainable Retrieval-Augmented Generation (ETRAG), which allows end-to-end optimization of the entire model, including the retriever, for the relation extraction objective by utilizing a differentiable selection of the $k$ nearest instances. We evaluate the relation extraction performance of ETRAG on the TACRED dataset, which is a standard benchmark for relation extraction. ETRAG demonstrates consistent improvements against the baseline model as retrieved instances are added. Furthermore, the analysis of instances retrieved by the end-to-end trained retriever confirms that the retrieved instances contain common relation labels or entities with the query and are specialized for the target task. Our findings provide a promising foundation for future research on retrieval-augmented generation and the broader applications of text generation in Natural Language Processing.