Exploiting Cross-Sentence Context for Neural Machine Translation

TL;DR

This paper introduces a cross-sentence context-aware method for neural machine translation that leverages historical document information to improve translation quality, achieving significant BLEU score improvements.

Contribution

It proposes a hierarchical summarization of historical context and two strategies for integrating this context into NMT, enhancing translation performance.

Findings

Up to +2.1 BLEU points improvement on Chinese-English translation

Hierarchical historical context effectively improves translation accuracy

Two strategies for context integration outperform baseline models

Abstract

In translation, considering the document as a whole can help to resolve ambiguities and inconsistencies. In this paper, we propose a cross-sentence context-aware approach and investigate the influence of historical contextual information on the performance of neural machine translation (NMT). First, this history is summarized in a hierarchical way. We then integrate the historical representation into NMT in two strategies: 1) a warm-start of encoder and decoder states, and 2) an auxiliary context source for updating decoder states. Experimental results on a large Chinese-English translation task show that our approach significantly improves upon a strong attention-based NMT system by up to +2.1 BLEU points.