Quality-Aware Decoding: Unifying Quality Estimation and Decoding

TL;DR

This paper introduces a novel token-level quality estimation model integrated into the decoding process of neural machine translation, significantly improving translation quality especially in document translation tasks.

Contribution

It presents a new token-level QE model and a decoding strategy that directly incorporates quality estimation, unifying quality assessment with decoding for the first time.

Findings

Improved translation quality over N-best list re-ranking

Up to 1.39x improvement in quality metrics

Significant benefits in document translation tasks

Abstract

Quality Estimation (QE) models for Neural Machine Translation (NMT) predict the quality of the hypothesis without having access to the reference. An emerging research direction in NMT involves the use of QE models, which have demonstrated high correlations with human judgment and can enhance translations through Quality-Aware Decoding. Although several approaches have been proposed based on sampling multiple candidate translations and picking the best candidate, none have integrated these models directly into the decoding process. In this paper, we address this by proposing a novel token-level QE model capable of reliably scoring partial translations. We build a uni-directional QE model for this, as decoder models are inherently trained and efficient on partial sequences. We then present a decoding strategy that integrates the QE model for Quality-Aware decoding and demonstrate that the translation quality improves when compared to the N-best list re-ranking with state-of-the-art QE models (up to $1.39$ XCOMET-XXL $\uparrow$). Finally, we show that our approach provides significant benefits in document translation tasks, where the quality of N-best lists is typically suboptimal. Code can be found at https://ai4lt.iar.kit.edu/english/projects\_kontextmt.php