Focused Concatenation for Context-Aware Neural Machine Translation

TL;DR

This paper introduces an improved context-aware neural machine translation method that emphasizes the current sentence during translation, enhancing translation quality and discourse coherence.

Contribution

The authors propose a novel concatenation technique that focuses on the current sentence and incorporates sentence boundary and distance information, outperforming existing methods.

Findings

Outperforms vanilla concatenation in translation quality

Improves handling of inter-sentential discourse phenomena

Strengthens sentence boundary recognition

Abstract

A straightforward approach to context-aware neural machine translation consists in feeding the standard encoder-decoder architecture with a window of consecutive sentences, formed by the current sentence and a number of sentences from its context concatenated to it. In this work, we propose an improved concatenation approach that encourages the model to focus on the translation of the current sentence, discounting the loss generated by target context. We also propose an additional improvement that strengthen the notion of sentence boundaries and of relative sentence distance, facilitating model compliance to the context-discounted objective. We evaluate our approach with both average-translation quality metrics and contrastive test sets for the translation of inter-sentential discourse phenomena, proving its superiority to the vanilla concatenation approach and other sophisticated context-aware systems.