Cross-Lingual Relation Extraction with Transformers

TL;DR

This paper introduces a Transformer-based cross-lingual relation extraction method that performs well without target language annotations or resources, enabling effective zero-shot transfer especially for low-resource languages.

Contribution

The paper presents a novel encoding scheme for Transformer models that encodes entity information, enabling zero-shot cross-lingual RE without additional data or resources.

Findings

Outperforms existing English RE models when trained on English data.

Achieves state-of-the-art zero-shot cross-lingual RE performance.

Effective for low-resource languages without extra training data.

Abstract

Relation extraction (RE) is one of the most important tasks in information extraction, as it provides essential information for many NLP applications. In this paper, we propose a cross-lingual RE approach that does not require any human annotation in a target language or any cross-lingual resources. Building upon unsupervised cross-lingual representation learning frameworks, we develop several deep Transformer based RE models with a novel encoding scheme that can effectively encode both entity location and entity type information. Our RE models, when trained with English data, outperform several deep neural network based English RE models. More importantly, our models can be applied to perform zero-shot cross-lingual RE, achieving the state-of-the-art cross-lingual RE performance on two datasets (68-89% of the accuracy of the supervised target-language RE model). The high cross-lingual transfer efficiency without requiring additional training data or cross-lingual resources shows that our RE models are especially useful for low-resource languages.