Cross-stitching Text and Knowledge Graph Encoders for Distantly Supervised Relation Extraction

TL;DR

This paper introduces cross-stitch bi-encoders that enable dynamic, full interaction between text and knowledge graph encoders, significantly improving distantly-supervised relation extraction performance across multiple domains.

Contribution

The paper proposes a novel cross-stitch mechanism for bi-encoders, allowing full and dynamic sharing of information between text and KG encoders, which was not possible in previous models.

Findings

Significant performance improvements on two relation extraction benchmarks.

Effective dynamic sharing of representations between encoders.

Enhanced cross-domain relation extraction accuracy.

Abstract

Bi-encoder architectures for distantly-supervised relation extraction are designed to make use of the complementary information found in text and knowledge graphs (KG). However, current architectures suffer from two drawbacks. They either do not allow any sharing between the text encoder and the KG encoder at all, or, in case of models with KG-to-text attention, only share information in one direction. Here, we introduce cross-stitch bi-encoders, which allow full interaction between the text encoder and the KG encoder via a cross-stitch mechanism. The cross-stitch mechanism allows sharing and updating representations between the two encoders at any layer, with the amount of sharing being dynamically controlled via cross-attention-based gates. Experimental results on two relation extraction benchmarks from two different domains show that enabling full interaction between the two encoders yields strong improvements.