Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision

TL;DR

This paper introduces a novel cross-lingual joint representation learning method for words and entities that leverages distant supervision from multilingual knowledge bases, improving cross-lingual NLP tasks without requiring parallel corpora.

Contribution

It proposes a new approach that captures complementary knowledge and aligns cross-lingual words and entities using regularizers and attention mechanisms, without needing parallel data.

Findings

Improves word translation accuracy

Enhances cross-lingual entity linking performance

Demonstrates effectiveness across multiple NLP tasks

Abstract

Joint representation learning of words and entities benefits many NLP tasks, but has not been well explored in cross-lingual settings. In this paper, we propose a novel method for joint representation learning of cross-lingual words and entities. It captures mutually complementary knowledge, and enables cross-lingual inferences among knowledge bases and texts. Our method does not require parallel corpora, and automatically generates comparable data via distant supervision using multi-lingual knowledge bases. We utilize two types of regularizers to align cross-lingual words and entities, and design knowledge attention and cross-lingual attention to further reduce noises. We conducted a series of experiments on three tasks: word translation, entity relatedness, and cross-lingual entity linking. The results, both qualitatively and quantitatively, demonstrate the significance of our method.