Pairwise Neural Machine Translation Evaluation

TL;DR

This paper introduces a neural network-based pairwise evaluation framework for machine translation quality, leveraging embeddings and modeling interactions to match human judgment.

Contribution

It proposes a novel, flexible neural approach for pairwise MT evaluation that effectively models interactions using compact embeddings.

Findings

Achieves correlation with human judgments comparable to state-of-the-art methods

Uses learned embeddings to capture lexical, syntactic, and semantic information

Provides an efficient and adaptable evaluation framework

Abstract

We present a novel framework for machine translation evaluation using neural networks in a pairwise setting, where the goal is to select the better translation from a pair of hypotheses, given the reference translation. In this framework, lexical, syntactic and semantic information from the reference and the two hypotheses is compacted into relatively small distributed vector representations, and fed into a multi-layer neural network that models the interaction between each of the hypotheses and the reference, as well as between the two hypotheses. These compact representations are in turn based on word and sentence embeddings, which are learned using neural networks. The framework is flexible, allows for efficient learning and classification, and yields correlation with humans that rivals the state of the art.