LLM with Relation Classifier for Document-Level Relation Extraction

TL;DR

This paper presents a novel classifier-LLM approach for document-level relation extraction that improves LLM focus on relevant entity pairs, significantly enhancing performance over previous LLM-based methods and narrowing the gap with BERT-based models.

Contribution

Introduces a classifier-LLM pipeline that directs LLM attention to potential relation pairs, improving document-level relation extraction accuracy.

Findings

Our method outperforms recent LLM-based DocRE models.

It narrows the performance gap with BERT-based models.

Experiments on benchmarks validate the effectiveness of the approach.

Abstract

Large language models (LLMs) have created a new paradigm for natural language processing. Despite their advancement, LLM-based methods still lag behind traditional approaches in document-level relation extraction (DocRE), a critical task for understanding complex entity relations within long context. This paper investigates the causes of this performance gap, identifying the dispersion of attention by LLMs due to entity pairs without relations as a key factor. We then introduce a novel classifier-LLM approach to DocRE. Particularly, the proposed approach begins with a classifier designed to select entity pair candidates that exhibit potential relations and then feed them to LLM for final relation classification. This method ensures that the LLM's attention is directed at relation-expressing entity pairs instead of those without relations during inference. Experiments on DocRE benchmarks reveal that our method significantly outperforms recent LLM-based DocRE models and narrows the performance gap with state-of-the-art BERT-based models.