Analysing Zero-Shot Readability-Controlled Sentence Simplification

TL;DR

This paper investigates the use of instruction-tuned large language models for zero-shot readability-controlled sentence simplification, highlighting challenges in achieving desired readability levels and the need for better evaluation metrics.

Contribution

It explores the potential of large language models for zero-shot sentence simplification based on readability control, without relying on parallel datasets.

Findings

Models struggle to simplify sentences to the lowest readability levels.

Standard automatic metrics often misjudge the quality of simplifications.

Source sentence characteristics hinder effective rewriting.

Abstract

Readability-controlled text simplification (RCTS) rewrites texts to lower readability levels while preserving their meaning. RCTS models often depend on parallel corpora with readability annotations on both source and target sides. Such datasets are scarce and difficult to curate, especially at the sentence level. To reduce reliance on parallel data, we explore using instruction-tuned large language models for zero-shot RCTS. Through automatic and manual evaluations, we examine: (1) how different types of contextual information affect a model's ability to generate sentences with the desired readability, and (2) the trade-off between achieving target readability and preserving meaning. Results show that all tested models struggle to simplify sentences (especially to the lowest levels) due to models' limitations and characteristics of the source sentences that impede adequate rewriting. Our experiments also highlight the need for better automatic evaluation metrics tailored to RCTS, as standard ones often misinterpret common simplification operations, and inaccurately assess readability and meaning preservation.