Multilingual Unsupervised Neural Machine Translation with Denoising Adapters

TL;DR

This paper introduces denoising adapters on pre-trained mBART-50 for multilingual unsupervised translation, offering a modular, efficient alternative to back-translation that supports incremental language addition with comparable translation quality.

Contribution

It proposes a novel adapter-based approach with denoising objectives for multilingual unsupervised translation, enhancing modularity and incremental language support.

Findings

Denoising adapters achieve BLEU scores comparable to back-translation.

The approach allows incremental addition of unseen languages.

Translation quality is maintained while reducing computational costs.

Abstract

We consider the problem of multilingual unsupervised machine translation, translating to and from languages that only have monolingual data by using auxiliary parallel language pairs. For this problem the standard procedure so far to leverage the monolingual data is back-translation, which is computationally costly and hard to tune. In this paper we propose instead to use denoising adapters, adapter layers with a denoising objective, on top of pre-trained mBART-50. In addition to the modularity and flexibility of such an approach we show that the resulting translations are on-par with back-translating as measured by BLEU, and furthermore it allows adding unseen languages incrementally.