Policy-based Sentence Simplification: Replacing Parallel Corpora with LLM-as-a-Judge

TL;DR

This paper presents a novel approach using Large Language Model-as-a-Judge to automatically generate policy-aligned training data for sentence simplification, eliminating the need for parallel corpora and enabling adaptable, policy-driven simplification systems.

Contribution

It introduces a method that leverages LLMs as judges to create training data aligned with specific simplification policies, improving flexibility and reducing reliance on costly human annotations.

Findings

Small open-source LLMs outperform GPT-4o on lexical simplification.

The approach achieves comparable results to GPT-4o on sentence rewriting.

The method demonstrates robustness across different model sizes and families.

Abstract

Sentence simplification aims to modify a sentence to make it easier to read and understand while preserving the meaning. Different applications require distinct simplification policies, such as replacing only complex words at the lexical level or rewriting the entire sentence while trading off details for simplicity. However, achieving such policy-driven control remains an open challenge. In this work, we introduce a simple yet powerful approach that leverages Large Language Model-as-a-Judge (LLM-as-a-Judge) to automatically construct policy-aligned training data, completely removing the need for costly human annotation or parallel corpora. Our method enables building simplification systems that adapt to diverse simplification policies. Remarkably, even small-scale open-source LLMs such as Phi-3-mini-3.8B surpass GPT-4o on lexical-oriented simplification, while achieving comparable performance on overall rewriting, as verified by both automatic metrics and human evaluations. The consistent improvements across model families and sizes demonstrate the robustness of our approach.