Iterative Refinement for Machine Translation

TL;DR

This paper introduces an iterative refinement approach for machine translation that revisits and improves previous translation decisions, leading to modest BLEU score improvements over traditional monotonic decoding methods.

Contribution

It proposes a neural network-based iterative refinement scheme that allows revisiting earlier translation decisions, unlike traditional monotonic decoding algorithms.

Findings

Up to 0.4 BLEU improvement on WMT15 German-English translation

Neural model predicts discrete substitutions based on attention mechanisms

Iterative refinement enhances phrase-based translation outputs

Abstract

Existing machine translation decoding algorithms generate translations in a strictly monotonic fashion and never revisit previous decisions. As a result, earlier mistakes cannot be corrected at a later stage. In this paper, we present a translation scheme that starts from an initial guess and then makes iterative improvements that may revisit previous decisions. We parameterize our model as a convolutional neural network that predicts discrete substitutions to an existing translation based on an attention mechanism over both the source sentence as well as the current translation output. By making less than one modification per sentence, we improve the output of a phrase-based translation system by up to 0.4 BLEU on WMT15 German-English translation.