Towards Neural Phrase-based Machine Translation

TL;DR

This paper introduces Neural Phrase-based Machine Translation (NPMT), which models phrase structures explicitly using Sleep-WAke Networks and achieves efficient, high-quality translation without attention mechanisms.

Contribution

It proposes a novel phrase-based translation model that directly outputs phrases, incorporates local reordering, and operates in linear time, outperforming existing neural methods.

Findings

NPMT outperforms strong NMT baselines on IWSLT datasets.

The model produces meaningful and coherent phrases.

Decodes in linear time without attention mechanisms.

Abstract

In this paper, we present Neural Phrase-based Machine Translation (NPMT). Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence modeling method. To mitigate the monotonic alignment requirement of SWAN, we introduce a new layer to perform (soft) local reordering of input sequences. Different from existing neural machine translation (NMT) approaches, NPMT does not use attention-based decoding mechanisms. Instead, it directly outputs phrases in a sequential order and can decode in linear time. Our experiments show that NPMT achieves superior performances on IWSLT 2014 German-English/English-German and IWSLT 2015 English-Vietnamese machine translation tasks compared with strong NMT baselines. We also observe that our method produces meaningful phrases in output languages.