Findings of the LoResMT 2021 Shared Task on COVID and Sign Language for Low-resource Languages

TL;DR

The LoResMT 2021 shared task evaluated machine translation systems for COVID-19 related data across low-resource spoken and sign languages, providing publicly available parallel corpora and benchmarking system performances.

Contribution

This paper introduces a shared task focusing on low-resource language translation for COVID-19 data, with new parallel corpora and evaluation results from multiple teams.

Findings

Maximum BLEU score of 36.0 for English--Irish

Maximum BLEU score of 34.6 for Irish--English

Maximum BLEU score of 24.2 for English--Marathi

Abstract

We present the findings of the LoResMT 2021 shared task which focuses on machine translation (MT) of COVID-19 data for both low-resource spoken and sign languages. The organization of this task was conducted as part of the fourth workshop on technologies for machine translation of low resource languages (LoResMT). Parallel corpora is presented and publicly available which includes the following directions: English$\leftrightarrow$Irish, English$\leftrightarrow$Marathi, and Taiwanese Sign language$\leftrightarrow$Traditional Chinese. Training data consists of 8112, 20933 and 128608 segments, respectively. There are additional monolingual data sets for Marathi and English that consist of 21901 segments. The results presented here are based on entries from a total of eight teams. Three teams submitted systems for English$\leftrightarrow$Irish while five teams submitted systems for English$\leftrightarrow$Marathi. Unfortunately, there were no systems submissions for the Taiwanese Sign language$\leftrightarrow$Traditional Chinese task. Maximum system performance was computed using BLEU and follow as 36.0 for English--Irish, 34.6 for Irish--English, 24.2 for English--Marathi, and 31.3 for Marathi--English.