Information-Propogation-Enhanced Neural Machine Translation by Relation Model

TL;DR

This paper introduces a relation network into neural machine translation to improve long-distance information flow, significantly enhancing translation quality over existing models.

Contribution

It proposes a novel relation network-enhanced NMT framework that better captures long-term dependencies, addressing limitations of RNNs and CNNs in sequence modeling.

Findings

Outperforms statistical MT models on two datasets.

Surpasses state-of-the-art NMT models in translation quality.

Enhances information propagation in neural networks.

Abstract

Even though sequence-to-sequence neural machine translation (NMT) model have achieved state-of-art performance in the recent fewer years, but it is widely concerned that the recurrent neural network (RNN) units are very hard to capture the long-distance state information, which means RNN can hardly find the feature with long term dependency as the sequence becomes longer. Similarly, convolutional neural network (CNN) is introduced into NMT for speeding recently, however, CNN focus on capturing the local feature of the sequence; To relieve this issue, we incorporate a relation network into the standard encoder-decoder framework to enhance information-propogation in neural network, ensuring that the information of the source sentence can flow into the decoder adequately. Experiments show that proposed framework outperforms the statistical MT model and the state-of-art NMT model significantly on two data sets with different scales.