Addressing the Rare Word Problem in Neural Machine Translation

TL;DR

This paper presents a novel method to improve neural machine translation by effectively handling rare and out-of-vocabulary words through alignment-based post-processing, significantly boosting translation quality.

Contribution

It introduces an alignment-based technique that allows NMT systems to better translate rare words by leveraging source-target alignments and dictionary lookups.

Findings

Achieved up to 2.8 BLEU point improvement over baseline NMT.

First NMT system to surpass the best WMT14 result with 37.5 BLEU.

Effective handling of OOV words enhances translation accuracy.

Abstract

Neural Machine Translation (NMT) is a new approach to machine translation that has shown promising results that are comparable to traditional approaches. A significant weakness in conventional NMT systems is their inability to correctly translate very rare words: end-to-end NMTs tend to have relatively small vocabularies with a single unk symbol that represents every possible out-of-vocabulary (OOV) word. In this paper, we propose and implement an effective technique to address this problem. We train an NMT system on data that is augmented by the output of a word alignment algorithm, allowing the NMT system to emit, for each OOV word in the target sentence, the position of its corresponding word in the source sentence. This information is later utilized in a post-processing step that translates every OOV word using a dictionary. Our experiments on the WMT14 English to French translation task show that this method provides a substantial improvement of up to 2.8 BLEU points over an equivalent NMT system that does not use this technique. With 37.5 BLEU points, our NMT system is the first to surpass the best result achieved on a WMT14 contest task.