Smarter edits? Post-editing with error highlights and translation suggestions

TL;DR

This study evaluates the effectiveness of LLM-derived error highlights and correction suggestions in post-editing translations, finding improved user experience but no significant gains in productivity or quality.

Contribution

It provides empirical evidence on the usefulness of LLM-based post-editing aids compared to traditional methods.

Findings

APE highlights were better received than QE-derived highlights

Correction suggestions improved user experience

No significant productivity or quality improvements observed

Abstract

As MT quality increases, interest in enhanced post-editing features such as QE-derived error highlights is growing, yet evidence for their usefulness remains limited. In this work, we explore the usefulness of LLM-derived error highlights and correction suggestions based on automatic post-editing (APE). We conduct a study where professional translators (En-Nl) post-edit translations using APE error highlights and correction suggestions and compare productivity, quality and user experience to regular PE and PE with QE-derived highlights. While no condition yielded productivity or quality gains compared to regular PE, APE highlights were better received than QE-derived highlights, and correction suggestions improved overall user experience.