A Densely Connected Criss-Cross Attention Network for Document-level Relation Extraction

TL;DR

This paper introduces Dense-CCNet, a novel model for document-level relation extraction that performs entity-pair-level logical reasoning using densely connected criss-cross attention, achieving state-of-the-art results.

Contribution

The paper proposes a new Dense-CCNet model that performs entity-pair-level reasoning with dense criss-cross attention, a novel approach in document-level RE.

Findings

Achieves state-of-the-art performance on DocRED, CDR, and GDA datasets.

Effectively captures multi-hop logical reasoning.

Enhances entity-pair representations through dense criss-cross attention.

Abstract

Document-level relation extraction (RE) aims to identify relations between two entities in a given document. Compared with its sentence-level counterpart, document-level RE requires complex reasoning. Previous research normally completed reasoning through information propagation on the mention-level or entity-level document-graph, but rarely considered reasoning at the entity-pair-level.In this paper, we propose a novel model, called Densely Connected Criss-Cross Attention Network (Dense-CCNet), for document-level RE, which can complete logical reasoning at the entity-pair-level. Specifically, the Dense-CCNet performs entity-pair-level logical reasoning through the Criss-Cross Attention (CCA), which can collect contextual information in horizontal and vertical directions on the entity-pair matrix to enhance the corresponding entity-pair representation. In addition, we densely connect multiple layers of the CCA to simultaneously capture the features of single-hop and multi-hop logical reasoning.We evaluate our Dense-CCNet model on three public document-level RE datasets, DocRED, CDR, and GDA. Experimental results demonstrate that our model achieves state-of-the-art performance on these three datasets.