Evaluating the fully automatic multi-language translation of the Swiss avalanche bulletin

TL;DR

This paper evaluates a fully automated, phrase-based translation system for Swiss avalanche bulletins across four languages, demonstrating comparable quality to manual translation and significant cost savings.

Contribution

It introduces a phrase-based automated translation system for avalanche bulletins and assesses its quality after two winter seasons, showing it is nearly equivalent to manual translation.

Findings

Participants rated system output similar to manual translations.

System accurately captured relevant avalanche situations.

Cost reduction offsets initial development expenses.

Abstract

The Swiss avalanche bulletin is produced twice a day in four languages. Due to the lack of time available for manual translation, a fully automated translation system is employed, based on a catalogue of predefined phrases and predetermined rules of how these phrases can be combined to produce sentences. The system is able to automatically translate such sentences from German into the target languages French, Italian and English without subsequent proofreading or correction. Our catalogue of phrases is limited to a small sublanguage. The reduction of daily translation costs is expected to offset the initial development costs within a few years. After being operational for two winter seasons, we assess here the quality of the produced texts based on an evaluation where participants rate real danger descriptions from both origins, the catalogue of phrases versus the manually written and translated texts. With a mean recognition rate of 55%, users can hardly distinguish between the two types of texts, and give similar ratings with respect to their language quality. Overall, the output from the catalogue system can be considered virtually equivalent to a text written by avalanche forecasters and then manually translated by professional translators. Furthermore, forecasters declared that all relevant situations were captured by the system with sufficient accuracy and within the limited time available.