Simplicity Level Estimate (SLE): A Learned Reference-Less Metric for Sentence Simplification

TL;DR

The paper introduces SLE, a learned, reference-less metric for sentence simplification that accurately assesses simplicity, overcoming limitations of existing metrics that rely on references and conflate attributes.

Contribution

It presents a novel learned evaluation metric (SLE) that specifically measures simplicity without needing references, improving correlation with human judgments.

Findings

SLE outperforms existing metrics in correlating with human judgments

SLE does not require multiple high-quality references

SLE effectively isolates simplicity as a distinct attribute

Abstract

Automatic evaluation for sentence simplification remains a challenging problem. Most popular evaluation metrics require multiple high-quality references -- something not readily available for simplification -- which makes it difficult to test performance on unseen domains. Furthermore, most existing metrics conflate simplicity with correlated attributes such as fluency or meaning preservation. We propose a new learned evaluation metric (SLE) which focuses on simplicity, outperforming almost all existing metrics in terms of correlation with human judgements.