Prompt-based Learning for Text Readability Assessment

TL;DR

This paper explores adapting pre-trained seq2seq models like T5 and BART for text readability assessment, demonstrating high accuracy and offering insights into prompt design and training strategies for improved cross-domain performance.

Contribution

It introduces a novel prompt-based, pairwise ranking approach using seq2seq models for readability assessment, enhancing cross-domain generalization and providing practical training tips.

Findings

Achieved 99.6% accuracy on Newsela dataset.

Identified prompt formats significantly impact model performance.

Demonstrated effective use of multiple data sources for training.

Abstract

We propose the novel adaptation of a pre-trained seq2seq model for readability assessment. We prove that a seq2seq model - T5 or BART - can be adapted to discern which text is more difficult from two given texts (pairwise). As an exploratory study to prompt-learn a neural network for text readability in a text-to-text manner, we report useful tips for future work in seq2seq training and ranking-based approach to readability assessment. Specifically, we test nine input-output formats/prefixes and show that they can significantly influence the final model performance. Also, we argue that the combination of text-to-text training and pairwise ranking setup 1) enables leveraging multiple parallel text simplification data for teaching readability and 2) trains a neural model for the general concept of readability (therefore, better cross-domain generalization). At last, we report a 99.6% pairwise classification accuracy on Newsela and a 98.7% for OneStopEnglish, through a joint training approach.