Multi$^2$OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT

TL;DR

Multi$^2$OIE introduces a multilingual open information extraction model that leverages BERT and multi-head attention, achieving superior performance and efficiency on benchmark datasets across multiple languages.

Contribution

The paper presents a novel sequence-labeling open IE model combining BERT with multi-head attention, extending to multilingual extraction without target language training data.

Findings

Outperforms existing sequence-labeling open IE systems in efficiency and accuracy.

Effective multilingual open IE on Spanish and Portuguese without language-specific training.

Achieves state-of-the-art results on benchmark datasets.

Abstract

In this paper, we propose Multi$^2$OIE, which performs open information extraction (open IE) by combining BERT with multi-head attention. Our model is a sequence-labeling system with an efficient and effective argument extraction method. We use a query, key, and value setting inspired by the Multimodal Transformer to replace the previously used bidirectional long short-term memory architecture with multi-head attention. Multi$^2$OIE outperforms existing sequence-labeling systems with high computational efficiency on two benchmark evaluation datasets, Re-OIE2016 and CaRB. Additionally, we apply the proposed method to multilingual open IE using multilingual BERT. Experimental results on new benchmark datasets introduced for two languages (Spanish and Portuguese) demonstrate that our model outperforms other multilingual systems without training data for the target languages.