Paying Attention to Multi-Word Expressions in Neural Machine Translation

TL;DR

This paper investigates how neural machine translation systems handle multi-word expressions and proposes two methods to improve translation accuracy by augmenting training data with MWEs, resulting in measurable BLEU score improvements.

Contribution

It introduces two novel strategies for enhancing NMT translation of MWEs by incorporating automatically extracted MWEs into training data, with demonstrated performance gains.

Findings

Adding MWE pairs to training data increased BLEU scores by 0.99 points.

Including full sentences with MWEs yielded minimal improvements.

Open-source tools for MWE extraction and alignment inspection are provided.

Abstract

Processing of multi-word expressions (MWEs) is a known problem for any natural language processing task. Even neural machine translation (NMT) struggles to overcome it. This paper presents results of experiments on investigating NMT attention allocation to the MWEs and improving automated translation of sentences that contain MWEs in English->Latvian and English->Czech NMT systems. Two improvement strategies were explored -(1) bilingual pairs of automatically extracted MWE candidates were added to the parallel corpus used to train the NMT system, and (2) full sentences containing the automatically extracted MWE candidates were added to the parallel corpus. Both approaches allowed to increase automated evaluation results. The best result - 0.99 BLEU point increase - has been reached with the first approach, while with the second approach minimal improvements achieved. We also provide open-source software and tools used for MWE extraction and alignment inspection.