Is MT Ready for the Next Crisis or Pandemic?

TL;DR

This paper evaluates the effectiveness of commercial machine translation systems in translating pandemic-related content for low-resource languages during crises, highlighting current limitations and readiness levels.

Contribution

It provides an empirical assessment of four commercial MT systems' performance on pandemic-related data in low-resource languages, focusing on crisis communication.

Findings

MT systems show limited accuracy in low-resource, crisis-related translation tasks

Current MT tools are not fully ready for critical pandemic communication needs

The study highlights gaps in translation quality for high-priority languages during crises

Abstract

Communication in times of crisis is essential. However, there is often a mismatch between the language of governments, aid providers, doctors, and those to whom they are providing aid. Commercial MT systems are reasonable tools to turn to in these scenarios. But how effective are these tools for translating to and from low resource languages, particularly in the crisis or medical domain? In this study, we evaluate four commercial MT systems using the TICO-19 dataset, which is composed of pandemic-related sentences from a large set of high priority languages spoken by communities most likely to be affected adversely in the next pandemic. We then assess the current degree of ``readiness'' for another pandemic (or epidemic) based on the usability of the output translations.