Unsupervised Neural Machine Translation

TL;DR

This paper introduces a fully unsupervised neural machine translation method that trains on monolingual data alone, achieving competitive translation quality without requiring parallel corpora.

Contribution

It presents a novel unsupervised NMT approach that relies solely on monolingual data, eliminating the need for parallel datasets and simplifying training.

Findings

Achieves 15.56 BLEU on French-English translation without parallel data.

Attains 10.21 BLEU on German-English translation without parallel data.

Improves to 21.81 and 15.24 BLEU when combined with small parallel corpora.

Abstract

In spite of the recent success of neural machine translation (NMT) in standard benchmarks, the lack of large parallel corpora poses a major practical problem for many language pairs. There have been several proposals to alleviate this issue with, for instance, triangulation and semi-supervised learning techniques, but they still require a strong cross-lingual signal. In this work, we completely remove the need of parallel data and propose a novel method to train an NMT system in a completely unsupervised manner, relying on nothing but monolingual corpora. Our model builds upon the recent work on unsupervised embedding mappings, and consists of a slightly modified attentional encoder-decoder model that can be trained on monolingual corpora alone using a combination of denoising and backtranslation. Despite the simplicity of the approach, our system obtains 15.56 and 10.21 BLEU points in WMT 2014 French-to-English and German-to-English translation. The model can also profit from small parallel corpora, and attains 21.81 and 15.24 points when combined with 100,000 parallel sentences, respectively. Our implementation is released as an open source project.