Context- and Sequence-Aware Convolutional Recurrent Encoder for Neural Machine Translation

TL;DR

This paper introduces a novel neural machine translation encoder that combines convolutional and recurrent layers to better capture context and sequential information, resulting in improved translation quality.

Contribution

The paper proposes a convolutional-recurrent encoder that integrates phrase-level context with sequential information, enhancing translation performance over existing models.

Findings

Achieved higher BLEU scores compared to state-of-the-art methods.

Demonstrated effective capture of phrase-level and sequential information.

Validated on German-English translation dataset.

Abstract

Neural Machine Translation model is a sequence-to-sequence converter based on neural networks. Existing models use recurrent neural networks to construct both the encoder and decoder modules. In alternative research, the recurrent networks were substituted by convolutional neural networks for capturing the syntactic structure in the input sentence and decreasing the processing time. We incorporate the goodness of both approaches by proposing a convolutional-recurrent encoder for capturing the context information as well as the sequential information from the source sentence. Word embedding and position embedding of the source sentence is performed prior to the convolutional encoding layer which is basically a n-gram feature extractor capturing phrase-level context information. The rectified output of the convolutional encoding layer is added to the original embedding vector, and the sum is normalized by layer normalization. The normalized output is given as a sequential input to the recurrent encoding layer that captures the temporal information in the sequence. For the decoder, we use the attention-based recurrent neural network. Translation task on the German-English dataset verifies the efficacy of the proposed approach from the higher BLEU scores achieved as compared to the state of the art.