MDQE: A More Accurate Direct Pretraining for Machine Translation Quality Estimation

TL;DR

This paper introduces MDQE, a novel pretraining framework for Machine Translation Quality Estimation that generates pseudo data and pretrains estimators directly on it, leading to improved performance without relying on large pre-trained models.

Contribution

It proposes a new direct pretraining framework with a generator and estimator for QE, reducing reliance on external pretraining models like BERT.

Findings

Outperforms existing QE methods on benchmark datasets.

Does not require pretraining models such as BERT.

Achieves higher accuracy in quality estimation tasks.

Abstract

It is expensive to evaluate the results of Machine Translation(MT), which usually requires manual translation as a reference. Machine Translation Quality Estimation (QE) is a task of predicting the quality of machine translations without relying on any reference. Recently, the emergence of predictor-estimator framework which trains the predictor as a feature extractor and estimator as a QE predictor, and pre-trained language models(PLM) have achieved promising QE performance. However, we argue that there are still gaps between the predictor and the estimator in both data quality and training objectives, which preclude QE models from benefiting from a large number of parallel corpora more directly. Based on previous related work that have alleviated gaps to some extent, we propose a novel framework that provides a more accurate direct pretraining for QE tasks. In this framework, a generator is trained to produce pseudo data that is closer to the real QE data, and a estimator is pretrained on these data with novel objectives that are the same as the QE task. Experiments on widely used benchmarks show that our proposed framework outperforms existing methods, without using any pretraining models such as BERT.