Document-Level Text Simplification: Dataset, Criteria and Baseline

TL;DR

This paper introduces the novel task of document-level text simplification, creating a large dataset, proposing a new evaluation metric, and establishing baseline models to advance research in this area.

Contribution

It defines the document-level simplification task, constructs the D-Wikipedia dataset, and proposes the D-SARI evaluation metric, filling gaps in current sentence-level simplification research.

Findings

D-Wikipedia dataset is reliable based on analysis and human evaluation.

D-SARI is more suitable for document-level evaluation.

Baseline models reveal current shortcomings in document-level simplification.

Abstract

Text simplification is a valuable technique. However, current research is limited to sentence simplification. In this paper, we define and investigate a new task of document-level text simplification, which aims to simplify a document consisting of multiple sentences. Based on Wikipedia dumps, we first construct a large-scale dataset named D-Wikipedia and perform analysis and human evaluation on it to show that the dataset is reliable. Then, we propose a new automatic evaluation metric called D-SARI that is more suitable for the document-level simplification task. Finally, we select several representative models as baseline models for this task and perform automatic evaluation and human evaluation. We analyze the results and point out the shortcomings of the baseline models.