Character-based Neural Machine Translation

TL;DR

This paper presents a character-based neural machine translation model that processes input and output as character sequences, enabling better handling of unseen words and reducing preprocessing needs, achieving comparable results to word-based models.

Contribution

The paper introduces a novel character-level NMT model that composes word representations from characters and generates words character-by-character, improving handling of unseen words.

Findings

Achieves translation quality comparable to word-based models.

Handles unseen words effectively due to character-level processing.

Reduces reliance on tokenization and preprocessing.

Abstract

We introduce a neural machine translation model that views the input and output sentences as sequences of characters rather than words. Since word-level information provides a crucial source of bias, our input model composes representations of character sequences into representations of words (as determined by whitespace boundaries), and then these are translated using a joint attention/translation model. In the target language, the translation is modeled as a sequence of word vectors, but each word is generated one character at a time, conditional on the previous character generations in each word. As the representation and generation of words is performed at the character level, our model is capable of interpreting and generating unseen word forms. A secondary benefit of this approach is that it alleviates much of the challenges associated with preprocessing/tokenization of the source and target languages. We show that our model can achieve translation results that are on par with conventional word-based models.