Character-based Neural Machine Translation

TL;DR

This paper introduces a character-based neural machine translation system that effectively handles large vocabularies and morphological complexity, leading to improved translation quality.

Contribution

It proposes a novel character-based embedding approach with convolutional and highway layers, replacing traditional word lookup methods in neural MT.

Findings

Achieved up to 3 BLEU point improvements on German-English translation.

Demonstrated effectiveness even for non-morphologically rich source languages.

Provided a scalable solution for large-vocabulary neural machine translation.

Abstract

Neural Machine Translation (MT) has reached state-of-the-art results. However, one of the main challenges that neural MT still faces is dealing with very large vocabularies and morphologically rich languages. In this paper, we propose a neural MT system using character-based embeddings in combination with convolutional and highway layers to replace the standard lookup-based word representations. The resulting unlimited-vocabulary and affix-aware source word embeddings are tested in a state-of-the-art neural MT based on an attention-based bidirectional recurrent neural network. The proposed MT scheme provides improved results even when the source language is not morphologically rich. Improvements up to 3 BLEU points are obtained in the German-English WMT task.