DORE: Document Ordered Relation Extraction based on Generative Framework

TL;DR

This paper introduces DORE, a generative framework for document-level relation extraction that generates ordered relation sequences, improving performance over previous models by addressing training paradigm issues.

Contribution

The paper proposes a new training paradigm for generative DocRE, including symbolic ordered sequence generation, parallel row processing, and negative sampling strategies.

Findings

Significant performance improvements on four datasets.

Effective handling of overlong sequences.

Enhanced training stability and accuracy.

Abstract

In recent years, there is a surge of generation-based information extraction work, which allows a more direct use of pre-trained language models and efficiently captures output dependencies. However, previous generative methods using lexical representation do not naturally fit document-level relation extraction (DocRE) where there are multiple entities and relational facts. In this paper, we investigate the root cause of the underwhelming performance of the existing generative DocRE models and discover that the culprit is the inadequacy of the training paradigm, instead of the capacities of the models. We propose to generate a symbolic and ordered sequence from the relation matrix which is deterministic and easier for model to learn. Moreover, we design a parallel row generation method to process overlong target sequences. Besides, we introduce several negative sampling strategies to improve the performance with balanced signals. Experimental results on four datasets show that our proposed method can improve the performance of the generative DocRE models. We have released our code at https://github.com/ayyyq/DORE.