Word Representation Models for Morphologically Rich Languages in Neural Machine Translation

TL;DR

This paper introduces new neural machine translation architectures that utilize character and morpheme-level word representations to better handle complex word forms in morphologically rich languages, resulting in improved translation quality.

Contribution

It proposes novel models that incorporate subword information into translation systems and jointly learn alignments and translations with a hard attention mechanism.

Findings

Achieved 1-1.5 BLEU point improvements over baselines.

Demonstrated effectiveness across multiple morphologically rich languages.

Enhanced handling of complex word forms in translation tasks.

Abstract

Dealing with the complex word forms in morphologically rich languages is an open problem in language processing, and is particularly important in translation. In contrast to most modern neural systems of translation, which discard the identity for rare words, in this paper we propose several architectures for learning word representations from character and morpheme level word decompositions. We incorporate these representations in a novel machine translation model which jointly learns word alignments and translations via a hard attention mechanism. Evaluating on translating from several morphologically rich languages into English, we show consistent improvements over strong baseline methods, of between 1 and 1.5 BLEU points.