BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation

TL;DR

BOUQuET is a comprehensive multilingual dataset and benchmark designed to improve translation quality evaluation across diverse languages, domains, and register levels, supporting collaborative and crowd-sourced translation research.

Contribution

It introduces a new multi-way, multi-domain, and multi-register dataset in 8 non-English languages, facilitating more accurate and inclusive translation assessments.

Findings

Broader domain representation compared to existing datasets

Enables crowd-sourced extension for diverse languages

Simplifies translation tasks for non-experts

Abstract

BOUQuET is a multi-way, multicentric and multi-register/domain dataset and benchmark, and a broader collaborative initiative. This dataset is handcrafted in 8 non-English languages. Each of these source languages are representative of the most widely spoken ones and therefore they have the potential to serve as pivot languages that will enable more accurate translations. The dataset is multicentric to enforce representation of multilingual language features. In addition, the dataset goes beyond the sentence level, as it is organized in paragraphs of various lengths. Compared with related machine translation datasets, we show that BOUQuET has a broader representation of domains while simplifying the translation task for non-experts. Therefore, BOUQuET is specially suitable for crowd-source extension for which we are launching a call aiming at collecting a multi-way parallel corpus covering any written language.