HanoiT: Enhancing Context-aware Translation via Selective Context

TL;DR

HanoiT introduces a layer-wise selection mechanism in neural machine translation to filter relevant context, significantly improving translation quality across multiple benchmarks by reducing noise from irrelevant words.

Contribution

The paper presents a novel end-to-end encoder-decoder model with a layer-wise selection mechanism for better utilization of document context in translation.

Findings

Model outperforms previous methods on four benchmarks

Selective context filtering reduces noise and improves translation accuracy

Layer-wise refinement enhances context relevance understanding

Abstract

Context-aware neural machine translation aims to use the document-level context to improve translation quality. However, not all words in the context are helpful. The irrelevant or trivial words may bring some noise and distract the model from learning the relationship between the current sentence and the auxiliary context. To mitigate this problem, we propose a novel end-to-end encoder-decoder model with a layer-wise selection mechanism to sift and refine the long document context. To verify the effectiveness of our method, extensive experiments and extra quantitative analysis are conducted on four document-level machine translation benchmarks. The experimental results demonstrate that our model significantly outperforms previous models on all datasets via the soft selection mechanism.