The Mediomatix Corpus: Parallel Data for Romansh Language Varieties via Comparable Schoolbooks

TL;DR

This paper introduces the Mediomatix Corpus, a large parallel dataset of Romansh language varieties derived from schoolbooks, enabling improved machine translation and NLP applications for this under-resourced language.

Contribution

It presents the first parallel corpus for Romansh idioms, created from schoolbooks, with automatic alignment and validation, facilitating NLP research and machine translation.

Findings

207,000 multi-parallel segments extracted

High-quality segments confirmed by human evaluation

Effective for training and evaluating machine translation models

Abstract

The five idioms (i.e., varieties) of the Romansh language are largely standardized and are taught in the schools of the respective communities in Switzerland. In this paper, we present the first parallel corpus of Romansh idioms. The corpus is based on 291 schoolbook volumes, which are comparable in content for the five idioms. We use automatic alignment methods to extract 207k multi-parallel segments from the books, with more than 2M tokens in total. A small-scale human evaluation confirms that the segments are highly parallel, making the dataset suitable for NLP applications such as machine translation between Romansh idioms. We release the parallel and unaligned versions of the dataset under a CC-BY-NC-SA license and demonstrate its utility for machine translation by training and evaluating an LLM and a supervised multilingual MT model on the dataset.