Multilingual Neural Machine Translation With Soft Decoupled Encoding

TL;DR

This paper introduces Soft Decoupled Encoding (SDE), a novel multilingual lexicon encoding framework that enhances neural machine translation for low-resource languages by sharing lexical information effectively without heuristic preprocessing.

Contribution

The paper proposes SDE, a new encoding method that represents words by spelling and shared semantic space, improving multilingual NMT performance on low-resource languages.

Findings

Achieved up to 2 BLEU point improvements on low-resource languages.

Set new state-of-the-art results on four low-resource language pairs.

Demonstrated consistent gains over strong multilingual NMT baselines.

Abstract

Multilingual training of neural machine translation (NMT) systems has led to impressive accuracy improvements on low-resource languages. However, there are still significant challenges in efficiently learning word representations in the face of paucity of data. In this paper, we propose Soft Decoupled Encoding (SDE), a multilingual lexicon encoding framework specifically designed to share lexical-level information intelligently without requiring heuristic preprocessing such as pre-segmenting the data. SDE represents a word by its spelling through a character encoding, and its semantic meaning through a latent embedding space shared by all languages. Experiments on a standard dataset of four low-resource languages show consistent improvements over strong multilingual NMT baselines, with gains of up to 2 BLEU on one of the tested languages, achieving the new state-of-the-art on all four language pairs.