Metric for Automatic Machine Translation Evaluation based on Universal Sentence Representations

TL;DR

This paper proposes a new metric for automatic machine translation evaluation using universal sentence representations trained on large-scale data, achieving state-of-the-art results on the WMT-2016 dataset.

Contribution

It introduces a novel evaluation metric leveraging universal sentence representations, demonstrating their effectiveness without relying on small-scale translation datasets.

Findings

Achieves state-of-the-art performance on WMT-2016 dataset

Universal sentence representations improve translation quality assessment

Method outperforms previous evaluation metrics

Abstract

Sentence representations can capture a wide range of information that cannot be captured by local features based on character or word N-grams. This paper examines the usefulness of universal sentence representations for evaluating the quality of machine translation. Although it is difficult to train sentence representations using small-scale translation datasets with manual evaluation, sentence representations trained from large-scale data in other tasks can improve the automatic evaluation of machine translation. Experimental results of the WMT-2016 dataset show that the proposed method achieves state-of-the-art performance with sentence representation features only.