Generative Neural Machine Translation

TL;DR

This paper presents Generative Neural Machine Translation (GNMT), a latent variable model that captures sentence semantics, improving translation quality especially with missing words and limited data, and enabling multilingual and semi-supervised translation.

Contribution

Introduces GNMT, a novel latent variable architecture for neural machine translation that enhances semantic modeling and supports multilingual and semi-supervised learning without extra parameters.

Findings

Achieves competitive BLEU scores on translation tasks.

Outperforms traditional models with missing source words.

Reduces overfitting with limited training data.

Abstract

We introduce Generative Neural Machine Translation (GNMT), a latent variable architecture which is designed to model the semantics of the source and target sentences. We modify an encoder-decoder translation model by adding a latent variable as a language agnostic representation which is encouraged to learn the meaning of the sentence. GNMT achieves competitive BLEU scores on pure translation tasks, and is superior when there are missing words in the source sentence. We augment the model to facilitate multilingual translation and semi-supervised learning without adding parameters. This framework significantly reduces overfitting when there is limited paired data available, and is effective for translating between pairs of languages not seen during training.