Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translation

TL;DR

This paper introduces a novel approach to Quality Estimation in Machine Translation by integrating uncertainty quantification features into a pre-trained language model, achieving state-of-the-art results.

Contribution

It extends QE to include uncertainty quantification from both black-box and glass-box perspectives, enhancing prediction accuracy.

Findings

Achieved state-of-the-art performance on WMT 2020 QE datasets.

Demonstrated the effectiveness of uncertainty features in QE.

Unified framework for black-box and glass-box QE features.

Abstract

Quality Estimation (QE) plays an essential role in applications of Machine Translation (MT). Traditionally, a QE system accepts the original source text and translation from a black-box MT system as input. Recently, a few studies indicate that as a by-product of translation, QE benefits from the model and training data's information of the MT system where the translations come from, and it is called the "glass-box QE". In this paper, we extend the definition of "glass-box QE" generally to uncertainty quantification with both "black-box" and "glass-box" approaches and design several features deduced from them to blaze a new trial in improving QE's performance. We propose a framework to fuse the feature engineering of uncertainty quantification into a pre-trained cross-lingual language model to predict the translation quality. Experiment results show that our method achieves state-of-the-art performances on the datasets of WMT 2020 QE shared task.