Enhanced Neural Machine Translation by Learning from Draft

TL;DR

This paper introduces a two-stage neural machine translation method that refines initial drafts by incorporating right-context information, leading to improved translation quality in Chinese-English tasks.

Contribution

It presents a novel draft-and-refine NMT framework that leverages a second attention-based system to enhance translation consistency by considering both input and draft.

Findings

Achieved 2.4 BLEU point improvement on small-scale task.

Achieved 0.9 BLEU point improvement on large-scale task.

Demonstrated effectiveness across different dataset sizes.

Abstract

Neural machine translation (NMT) has recently achieved impressive results. A potential problem of the existing NMT algorithm, however, is that the decoding is conducted from left to right, without considering the right context. This paper proposes an two-stage approach to solve the problem. In the first stage, a conventional attention-based NMT system is used to produce a draft translation, and in the second stage, a novel double-attention NMT system is used to refine the translation, by looking at the original input as well as the draft translation. This drafting-and-refinement can obtain the right-context information from the draft, hence producing more consistent translations. We evaluated this approach using two Chinese-English translation tasks, one with 44k pairs and 1M pairs respectively. The experiments showed that our approach achieved positive improvements over the conventional NMT system: the improvements are 2.4 and 0.9 BLEU points on the small-scale and large-scale tasks, respectively.