Modeling Past and Future for Neural Machine Translation

TL;DR

This paper introduces a novel neural machine translation mechanism that explicitly models translated and untranslated source contents separately, significantly improving translation quality and alignment accuracy across multiple language pairs.

Contribution

It proposes a new approach that separates source information into Past and Future contents using recurrent layers, enhancing translation and alignment performance.

Findings

Outperforms conventional coverage models in translation quality

Reduces alignment error rate in multiple language pairs

Improves translation performance in Chinese-English, German-English, English-German

Abstract

Existing neural machine translation systems do not explicitly model what has been translated and what has not during the decoding phase. To address this problem, we propose a novel mechanism that separates the source information into two parts: translated Past contents and untranslated Future contents, which are modeled by two additional recurrent layers. The Past and Future contents are fed to both the attention model and the decoder states, which offers NMT systems the knowledge of translated and untranslated contents. Experimental results show that the proposed approach significantly improves translation performance in Chinese-English, German-English and English-German translation tasks. Specifically, the proposed model outperforms the conventional coverage model in both of the translation quality and the alignment error rate.