Cross-lingual Word Embeddings beyond Zero-shot Machine Translation

TL;DR

This paper investigates how cross-lingual word embeddings enable transfer of translation knowledge to unseen languages, highlighting the influence of language relatedness and architectural limitations.

Contribution

It demonstrates the transferability of translation knowledge via cross-lingual embeddings to unseen languages and analyzes factors affecting transfer strength.

Findings

Transfer of translation knowledge is weak but possible.

Language relatedness impacts transfer effectiveness.

Multilingual architecture limitations hinder transferability.

Abstract

We explore the transferability of a multilingual neural machine translation model to unseen languages when the transfer is grounded solely on the cross-lingual word embeddings. Our experimental results show that the translation knowledge can transfer weakly to other languages and that the degree of transferability depends on the languages' relatedness. We also discuss the limiting aspects of the multilingual architectures that cause weak translation transfer and suggest how to mitigate the limitations.