German Text Simplification: Finetuning Large Language Models with Semi-Synthetic Data

TL;DR

This paper explores the use of synthetically generated data to train large language models for German text simplification, demonstrating significant improvements in simplifying real-world texts despite data scarcity.

Contribution

It introduces a novel approach of using GPT-4 generated synthetic data for finetuning large language models in German text simplification.

Findings

Models significantly simplify real-world texts

Synthetic data improves model performance

Current rule-based metrics have limitations

Abstract

This study pioneers the use of synthetically generated data for training generative models in document-level text simplification of German texts. We demonstrate the effectiveness of our approach with real-world online texts. Addressing the challenge of data scarcity in language simplification, we crawled professionally simplified German texts and synthesized a corpus using GPT-4. We finetune Large Language Models with up to 13 billion parameters on this data and evaluate their performance. This paper employs various methodologies for evaluation and demonstrates the limitations of currently used rule-based metrics. Both automatic and manual evaluations reveal that our models can significantly simplify real-world online texts, indicating the potential of synthetic data in improving text simplification.