Towards Fine-Grained Information: Identifying the Type and Location of Translation Errors

TL;DR

This paper introduces FG-TED, a model for detecting translation error types and locations simultaneously, improving accuracy and reliability especially in low-resource scenarios.

Contribution

The paper proposes a novel FG-TED task and model that identify both error type and position, addressing limitations of previous methods.

Findings

Achieves state-of-the-art results on the benchmark dataset.

Provides more reliable predictions in low-resource and transfer scenarios.

Constructs synthetic datasets to improve training and reduce labeling disagreements.

Abstract

Fine-grained information on translation errors is helpful for the translation evaluation community. Existing approaches can not synchronously consider error position and type, failing to integrate the error information of both. In this paper, we propose Fine-Grained Translation Error Detection (FG-TED) task, aiming at identifying both the position and the type of translation errors on given source-hypothesis sentence pairs. Besides, we build an FG-TED model to predict the \textbf{addition} and \textbf{omission} errors -- two typical translation accuracy errors. First, we use a word-level classification paradigm to form our model and use the shortcut learning reduction to relieve the influence of monolingual features. Besides, we construct synthetic datasets for model training, and relieve the disagreement of data labeling in authoritative datasets, making the experimental benchmark concordant. Experiments show that our model can identify both error type and position concurrently, and gives state-of-the-art results on the restored dataset. Our model also delivers more reliable predictions on low-resource and transfer scenarios than existing baselines. The related datasets and the source code will be released in the future.