Supervised Attentions for Neural Machine Translation

TL;DR

This paper introduces a supervised attention mechanism in neural machine translation that leverages true alignments during training, significantly enhancing translation and alignment quality over existing models.

Contribution

It proposes a novel method to incorporate true alignment information into neural machine translation training, improving accuracy and outperforming traditional syntax-based systems.

Findings

Improved translation quality on Chinese-English tasks

Enhanced alignment accuracy with supervised attention

Outperforms state-of-the-art syntax-based systems

Abstract

In this paper, we improve the attention or alignment accuracy of neural machine translation by utilizing the alignments of training sentence pairs. We simply compute the distance between the machine attentions and the "true" alignments, and minimize this cost in the training procedure. Our experiments on large-scale Chinese-to-English task show that our model improves both translation and alignment qualities significantly over the large-vocabulary neural machine translation system, and even beats a state-of-the-art traditional syntax-based system.