A Parallel Evaluation Data Set of Software Documentation with Document Structure Annotation

TL;DR

This paper introduces a parallel evaluation dataset for machine translation of software documentation, including structural annotations, covering multiple low-resource language pairs to enhance translation research and system tuning.

Contribution

It provides a novel dataset with structural document annotations for software documentation translation, expanding evaluation scenarios for low-resource language pairs.

Findings

Dataset includes English-Hindi, Indonesian, Malay, Thai translations.

Structural annotations enable context-aware translation evaluation.

Machine translation results demonstrate the dataset's utility.

Abstract

This paper accompanies the software documentation data set for machine translation, a parallel evaluation data set of data originating from the SAP Help Portal, that we released to the machine translation community for research purposes. It offers the possibility to tune and evaluate machine translation systems in the domain of corporate software documentation and contributes to the availability of a wider range of evaluation scenarios. The data set comprises of the language pairs English to Hindi, Indonesian, Malay and Thai, and thus also increases the test coverage for the many low-resource language pairs. Unlike most evaluation data sets that consist of plain parallel text, the segments in this data set come with additional metadata that describes structural information of the document context. We provide insights into the origin and creation, the particularities and characteristics of the data set as well as machine translation results.