ViBidirectionMT-Eval: Machine Translation for Vietnamese-Chinese and Vietnamese-Lao language pair

TL;DR

This paper reports on the VLSP 2022-2023 shared tasks for developing machine translation systems for Vietnamese-Chinese and Vietnamese-Lao, evaluated through BLEU scores and human judgments to assess translation quality.

Contribution

It introduces the shared tasks for Vietnamese-Chinese and Vietnamese-Lao translation, providing benchmark datasets and evaluation protocols for the community.

Findings

Systems achieved competitive BLEU scores

Human judgments correlated with automatic metrics

Benchmark datasets established for future research

Abstract

This paper presents an results of the VLSP 2022-2023 Machine Translation Shared Tasks, focusing on Vietnamese-Chinese and Vietnamese-Lao machine translation. The tasks were organized as part of the 9th, 10th annual workshop on Vietnamese Language and Speech Processing (VLSP 2022, VLSP 2023). The objective of the shared task was to build machine translation systems, specifically targeting Vietnamese-Chinese and Vietnamese-Lao translation (corresponding to 4 translation directions). The submission were evaluated on 1,000 pairs for testing (news and general domains) using established metrics like BLEU [11] and SacreBLEU [12]. Additionally, system outputs also were evaluated with human judgment provided by experts in Chinese and Lao languages. These human assessments played a crucial role in ranking the performance of the machine translation models, ensuring a more comprehensive evaluation.