LC-Score: Reference-less estimation of Text Comprehension Difficulty

TL;DR

This paper introduces LC-Score, a reference-less metric for estimating French text comprehension difficulty on a 0-100 scale, utilizing linguistic indicators and neural models, to improve accessibility and simplify text evaluation.

Contribution

The paper presents LC-Score, a novel reference-less French text comprehension metric trained with a proxy classification task, outperforming existing readability metrics.

Findings

Both models outperform FKGL in human evaluations.

Neural approach effectively captures comprehension difficulty.

Indicator-based model provides interpretable insights.

Abstract

Being able to read and understand written text is critical in a digital era. However, studies shows that a large fraction of the population experiences comprehension issues. In this context, further initiatives in accessibility are required to improve the audience text comprehension. However, writers are hardly assisted nor encouraged to produce easy-to-understand content. Moreover, Automatic Text Simplification (ATS) model development suffers from the lack of metric to accurately estimate comprehension difficulty We present \textsc{LC-Score}, a simple approach for training text comprehension metric for any French text without reference \ie predicting how easy to understand a given text is on a $[0, 100]$ scale. Our objective with this scale is to quantitatively capture the extend to which a text suits to the \textit{Langage Clair} (LC, \textit{Clear Language}) guidelines, a French initiative closely related to English Plain Language. We explore two approaches: (i) using linguistically motivated indicators used to train statistical models, and (ii) neural learning directly from text leveraging pre-trained language models. We introduce a simple proxy task for comprehension difficulty training as a classification task. To evaluate our models, we run two distinct human annotation experiments, and find that both approaches (indicator based and neural) outperforms commonly used readability and comprehension metrics such as FKGL.