Document-Level Neural Machine Translation with Hierarchical Attention Networks

TL;DR

This paper introduces a hierarchical attention model for document-level neural machine translation, effectively incorporating contextual information to improve translation quality.

Contribution

It proposes a novel hierarchical attention mechanism integrated into NMT, capturing document context in a structured and dynamic way, enhancing translation performance.

Findings

Hierarchical attention significantly improves BLEU scores.

Both encoder and decoder benefit from document context.

Achieves state-of-the-art results in context-aware NMT.

Abstract

Neural Machine Translation (NMT) can be improved by including document-level contextual information. For this purpose, we propose a hierarchical attention model to capture the context in a structured and dynamic manner. The model is integrated in the original NMT architecture as another level of abstraction, conditioning on the NMT model's own previous hidden states. Experiments show that hierarchical attention significantly improves the BLEU score over a strong NMT baseline with the state-of-the-art in context-aware methods, and that both the encoder and decoder benefit from context in complementary ways.