Findings of the WMT 2022 Shared Task on Translation Suggestion

TL;DR

This paper reports the results of the inaugural WMT shared task on Translation Suggestion, evaluating models' ability to suggest alternative words or phrases in machine-translated documents, with and without hints.

Contribution

It introduces a new shared task on translation suggestion, provides datasets for English-German and English-Chinese, and evaluates submissions using BLEU scores.

Findings

92 submissions for sub-task one

6 submissions for sub-task two

Most teams covered all translation directions

Abstract

We report the result of the first edition of the WMT shared task on Translation Suggestion (TS). The task aims to provide alternatives for specific words or phrases given the entire documents generated by machine translation (MT). It consists two sub-tasks, namely, the naive translation suggestion and translation suggestion with hints. The main difference is that some hints are provided in sub-task two, therefore, it is easier for the model to generate more accurate suggestions. For sub-task one, we provide the corpus for the language pairs English-German and English-Chinese. And only English-Chinese corpus is provided for the sub-task two. We received 92 submissions from 5 participating teams in sub-task one and 6 submissions for the sub-task 2, most of them covering all of the translation directions. We used the automatic metric BLEU for evaluating the performance of each submission.