Using Whole Document Context in Neural Machine Translation

TL;DR

This paper introduces a method to incorporate entire document context into neural machine translation models, improving translation coherence across sentences for multiple language pairs.

Contribution

It proposes a simple approach to add source document context with accurate boundaries to Transformer models, enhancing document-level translation performance.

Findings

Improved translation quality in English-German, English-French, and French-English tasks.

Model learns to use document context for better coherence.

Promising results demonstrating the effectiveness of document-level context inclusion.

Abstract

In Machine Translation, considering the document as a whole can help to resolve ambiguities and inconsistencies. In this paper, we propose a simple yet promising approach to add contextual information in Neural Machine Translation. We present a method to add source context that capture the whole document with accurate boundaries, taking every word into account. We provide this additional information to a Transformer model and study the impact of our method on three language pairs. The proposed approach obtains promising results in the English-German, English-French and French-English document-level translation tasks. We observe interesting cross-sentential behaviors where the model learns to use document-level information to improve translation coherence.