Continuous Space Reordering Models for Phrase-based MT

TL;DR

This paper introduces neural network-based reordering models for phrase-based machine translation, demonstrating improved translation quality over traditional methods and integrating reordering into neural encoder-decoder frameworks.

Contribution

It proposes neuralized lexicalized reordering and operation sequence models, showing their effectiveness and comparing them to POS/tag-based models, and explores integrating reordering into neural MT.

Findings

Neural reordering models improve BLEU scores by up to 0.6 points.

Neural models outperform POS/tag-based reordering models.

Explicit reordering triggers in neural MT did not improve results.

Abstract

Bilingual sequence models improve phrase-based translation and reordering by overcoming phrasal independence assumption and handling long range reordering. However, due to data sparsity, these models often fall back to very small context sizes. This problem has been previously addressed by learning sequences over generalized representations such as POS tags or word clusters. In this paper, we explore an alternative based on neural network models. More concretely we train neuralized versions of lexicalized reordering and the operation sequence models using feed-forward neural network. Our results show improvements of up to 0.6 and 0.5 BLEU points on top of the baseline German->English and English->German systems. We also observed improvements compared to the systems that used POS tags and word clusters to train these models. Because we modify the bilingual corpus to integrate reordering operations, this allows us to also train a sequence-to-sequence neural MT model having explicit reordering triggers. Our motivation was to directly enable reordering information in the encoder-decoder framework, which otherwise relies solely on the attention model to handle long range reordering. We tried both coarser and fine-grained reordering operations. However, these experiments did not yield any improvements over the baseline Neural MT systems.