Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation

TL;DR

This paper demonstrates that lexical data augmentation using bilingual lexica significantly improves unsupervised multilingual machine translation, especially when using carefully curated lexica, and introduces a new multilingual lexicon for low-resource languages.

Contribution

It introduces a practical approach to enhance unsupervised multilingual translation with lexical data augmentation and provides a new high-quality multilingual lexicon for low-resource languages.

Findings

Lexical data augmentation yields sizable translation improvements.

Different augmentation methods provide similar gains and can be combined.

Curated lexica outperform noisier, larger lexica, especially for bigger models.

Abstract

Neural machine translation (NMT) has progressed rapidly over the past several years, and modern models are able to achieve relatively high quality using only monolingual text data, an approach dubbed Unsupervised Machine Translation (UNMT). However, these models still struggle in a variety of ways, including aspects of translation that for a human are the easiest - for instance, correctly translating common nouns. This work explores a cheap and abundant resource to combat this problem: bilingual lexica. We test the efficacy of bilingual lexica in a real-world set-up, on 200-language translation models trained on web-crawled text. We present several findings: (1) using lexical data augmentation, we demonstrate sizable performance gains for unsupervised translation; (2) we compare several families of data augmentation, demonstrating that they yield similar improvements, and can be combined for even greater improvements; (3) we demonstrate the importance of carefully curated lexica over larger, noisier ones, especially with larger models; and (4) we compare the efficacy of multilingual lexicon data versus human-translated parallel data. Finally, we open-source GATITOS (available at https://github.com/google-research/url-nlp/tree/main/gatitos), a new multilingual lexicon for 26 low-resource languages, which had the highest performance among lexica in our experiments.