Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

TL;DR

This paper introduces a neural network model called RNN Encoder-Decoder that learns phrase representations to improve statistical machine translation by capturing semantic and syntactic information.

Contribution

The paper presents a novel RNN Encoder-Decoder model that enhances phrase representation learning for machine translation, improving translation quality.

Findings

Improved translation performance using RNN-based phrase probabilities

The model learns meaningful semantic and syntactic phrase representations

Enhancement of existing translation models with RNN-derived features

Abstract

In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed-length vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence. The performance of a statistical machine translation system is empirically found to improve by using the conditional probabilities of phrase pairs computed by the RNN Encoder-Decoder as an additional feature in the existing log-linear model. Qualitatively, we show that the proposed model learns a semantically and syntactically meaningful representation of linguistic phrases.