Document-level Neural Machine Translation with Document Embeddings

TL;DR

This paper introduces a document-aware neural machine translation method that leverages multiple forms of document embeddings to incorporate detailed context, significantly improving translation quality over existing models.

Contribution

It proposes a novel approach to utilize both global and local document embeddings in Transformer-based NMT, enhancing context modeling beyond previous methods.

Findings

Significant performance improvements over strong baselines

Effective modeling of deeper document-level context

Enhanced translation quality with document embeddings

Abstract

Standard neural machine translation (NMT) is on the assumption of document-level context independent. Most existing document-level NMT methods are satisfied with a smattering sense of brief document-level information, while this work focuses on exploiting detailed document-level context in terms of multiple forms of document embeddings, which is capable of sufficiently modeling deeper and richer document-level context. The proposed document-aware NMT is implemented to enhance the Transformer baseline by introducing both global and local document-level clues on the source end. Experiments show that the proposed method significantly improves the translation performance over strong baselines and other related studies.