Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation

TL;DR

This paper introduces DecSDE, a character n-gram based decoder embedding that enhances multilingual NMT for low-resource languages, especially improving translation into these languages.

Contribution

It proposes DecSDE, a novel decoder embedding method that improves multilingual transfer for low-resource language translation in NMT models.

Findings

Up to 1.8 BLEU improvement in English-to-LRL translation

DecSDE outperforms previous embedding methods

Effective for four different low-resource languages

Abstract

To improve the performance of Neural Machine Translation~(NMT) for low-resource languages~(LRL), one effective strategy is to leverage parallel data from a related high-resource language~(HRL). However, multilingual data has been found more beneficial for NMT models that translate from the LRL to a target language than the ones that translate into the LRLs. In this paper, we aim to improve the effectiveness of multilingual transfer for NMT models that translate \emph{into} the LRL, by designing a better decoder word embedding. Extending upon a general-purpose multilingual encoding method Soft Decoupled Encoding~\citep{SDE}, we propose DecSDE, an efficient character n-gram based embedding specifically designed for the NMT decoder. Our experiments show that DecSDE leads to consistent gains of up to 1.8 BLEU on translation from English to four different languages.