Neural Machine Translation with Extended Context

TL;DR

This paper explores the impact of extended context in neural machine translation, showing that larger context segments can improve coherence and are robust without degrading translation quality.

Contribution

It introduces methods for incorporating extended source and bilingual context in attention-based neural machine translation models.

Findings

Models learn to differentiate information from multiple segments.

Extended context improves textual coherence in some cases.

Models remain robust with respect to translation quality.

Abstract

We investigate the use of extended context in attention-based neural machine translation. We base our experiments on translated movie subtitles and discuss the effect of increasing the segments beyond single translation units. We study the use of extended source language context as well as bilingual context extensions. The models learn to distinguish between information from different segments and are surprisingly robust with respect to translation quality. In this pilot study, we observe interesting cross-sentential attention patterns that improve textual coherence in translation at least in some selected cases.