Align and Shine: Building High-Quality Sentence-Aligned Corpora for Multilingual Text Simplification

TL;DR

This paper presents a method for creating high-quality, multilingual sentence-aligned corpora for text simplification, addressing data scarcity in non-English languages.

Contribution

It introduces a novel approach for sentence-level alignment from comparable corpora and releases a multilingual dataset for text simplification.

Findings

Constructed a multilingual corpus for text simplification in five languages.

Developed mechanisms for sentence-level alignment from document-level data.

The dataset is publicly available for research use.

Abstract

Text simplification plays a crucial role in improving the accessibility and comprehensibility of written information for diverse audiences, including language learners and readers with limited literacy. Despite its importance, large-scale, high-quality datasets for training and evaluating text simplification models remain scarce for languages other than English. This paper reports an experimental study on the collection and processing of crowd-sourced simplification data from comparable corpora to construct a corpus suitable for both training and testing text simplification systems across multiple languages (Catalan, English, French, Italian and Spanish). We report mechanisms for sentence-level alignment from document-level data. The resulting dataset of the aligned sentence pairs is publicly available.