Predicting Human Translation Difficulty with Neural Machine Translation

TL;DR

This study investigates how neural machine translation-derived surprisal and attention metrics can predict human translation difficulty, revealing surprisal as the most effective predictor across multiple languages.

Contribution

It demonstrates that NMT-based surprisal and attention features effectively predict human translation difficulty, with surprisal being the strongest single predictor, across diverse language pairs.

Findings

Surprisal from NMT models is the best predictor of translation duration.

Surprisal and attention are complementary predictors.

The study covers data from hundreds of translators across 13 language pairs.

Abstract

Human translators linger on some words and phrases more than others, and predicting this variation is a step towards explaining the underlying cognitive processes. Using data from the CRITT Translation Process Research Database, we evaluate the extent to which surprisal and attentional features derived from a Neural Machine Translation (NMT) model account for reading and production times of human translators. We find that surprisal and attention are complementary predictors of translation difficulty, and that surprisal derived from a NMT model is the single most successful predictor of production duration. Our analyses draw on data from hundreds of translators operating across 13 language pairs, and represent the most comprehensive investigation of human translation difficulty to date.