Rethinking Document-level Neural Machine Translation

TL;DR

This paper demonstrates that the original Transformer model, with proper training, can effectively handle long document translation, outperforming many recent approaches across multiple datasets and metrics.

Contribution

It shows that the original Transformer, with appropriate training techniques, is sufficient for high-quality document-level translation without needing novel architectures.

Findings

Transformer achieves strong results on 2000-word documents

Document-level models outperform sentence-level models

The approach outperforms previous methods on multiple metrics

Abstract

This paper does not aim at introducing a novel model for document-level neural machine translation. Instead, we head back to the original Transformer model and hope to answer the following question: Is the capacity of current models strong enough for document-level translation? Interestingly, we observe that the original Transformer with appropriate training techniques can achieve strong results for document translation, even with a length of 2000 words. We evaluate this model and several recent approaches on nine document-level datasets and two sentence-level datasets across six languages. Experiments show that document-level Transformer models outperforms sentence-level ones and many previous methods in a comprehensive set of metrics, including BLEU, four lexical indices, three newly proposed assistant linguistic indicators, and human evaluation.