HistRED: A Historical Document-Level Relation Extraction Dataset

TL;DR

HistRED is a new bilingual dataset for historical document-level relation extraction, enabling research on Korean and Hanja texts with diverse context lengths, and demonstrating improved RE performance using multi-language information.

Contribution

We introduce HistRED, a novel dataset for historical relation extraction with bilingual annotations and variable text lengths, and propose a model leveraging both languages for better accuracy.

Findings

Our bilingual model outperforms monolingual baselines.

HistRED supports diverse context lengths for robust evaluation.

The dataset is publicly available for research use.

Abstract

Despite the extensive applications of relation extraction (RE) tasks in various domains, little has been explored in the historical context, which contains promising data across hundreds and thousands of years. To promote the historical RE research, we present HistRED constructed from Yeonhaengnok. Yeonhaengnok is a collection of records originally written in Hanja, the classical Chinese writing, which has later been translated into Korean. HistRED provides bilingual annotations such that RE can be performed on Korean and Hanja texts. In addition, HistRED supports various self-contained subtexts with different lengths, from a sentence level to a document level, supporting diverse context settings for researchers to evaluate the robustness of their RE models. To demonstrate the usefulness of our dataset, we propose a bilingual RE model that leverages both Korean and Hanja contexts to predict relations between entities. Our model outperforms monolingual baselines on HistRED, showing that employing multiple language contexts supplements the RE predictions. The dataset is publicly available at: https://huggingface.co/datasets/Soyoung/HistRED under CC BY-NC-ND 4.0 license.