Introducing Quality Estimation to Machine Translation Post-editing Workflow: An Empirical Study on Its Usefulness

TL;DR

This study evaluates the effectiveness of sentence-level Quality Estimation in English-Chinese Machine Translation Post-editing, showing it reduces editing time and supports multiple functions, though inaccuracies can hinder progress.

Contribution

It provides empirical evidence on QE's usefulness in MTPE, highlighting its impact on efficiency, validation, and potential limitations in translation workflows.

Findings

QE significantly reduces post-editing time

QE's benefits are consistent across MT quality levels and translator expertise

Inaccurate QE can negatively affect post-editing processes

Abstract

This preliminary study investigates the usefulness of sentence-level Quality Estimation (QE) in English-Chinese Machine Translation Post-Editing (MTPE), focusing on its impact on post-editing speed and student translators' perceptions. It also explores the interaction effects between QE and MT quality, as well as between QE and translation expertise. The findings reveal that QE significantly reduces post-editing time. The examined interaction effects were not significant, suggesting that QE consistently improves MTPE efficiency across medium- and high-quality MT outputs and among student translators with varying levels of expertise. In addition to indicating potentially problematic segments, QE serves multiple functions in MTPE, such as validating translators' evaluations of MT quality and enabling them to double-check translation outputs. However, interview data suggest that inaccurate QE may hinder post-editing processes. This research provides new insights into the strengths and limitations of QE, facilitating its more effective integration into MTPE workflows to enhance translators' productivity.