Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation

TL;DR

This paper evaluates the cross-lingual transfer capabilities of a massively multilingual NMT encoder across various tasks and languages, demonstrating its effectiveness compared to mBERT in zero-shot scenarios.

Contribution

It provides a comprehensive assessment of the encoder's cross-lingual transfer performance on multiple tasks and languages, highlighting its potential for multilingual NLP applications.

Findings

Gains in zero-shot transfer in 4 out of 5 tasks

Effective cross-lingual representations for diverse languages

Outperforms mBERT in several transfer scenarios

Abstract

The recently proposed massively multilingual neural machine translation (NMT) system has been shown to be capable of translating over 100 languages to and from English within a single model. Its improved translation performance on low resource languages hints at potential cross-lingual transfer capability for downstream tasks. In this paper, we evaluate the cross-lingual effectiveness of representations from the encoder of a massively multilingual NMT model on 5 downstream classification and sequence labeling tasks covering a diverse set of over 50 languages. We compare against a strong baseline, multilingual BERT (mBERT), in different cross-lingual transfer learning scenarios and show gains in zero-shot transfer in 4 out of these 5 tasks.