Learning To Split and Rephrase From Wikipedia Edit History

TL;DR

This paper introduces WikiSplit, a large dataset of Wikipedia sentence splits, enabling improved models for sentence splitting and rephrasing that outperform previous benchmarks.

Contribution

The creation of WikiSplit, a dataset with one million sentence rewrites from Wikipedia, significantly advancing training data availability for split and rephrase tasks.

Findings

Models trained on WikiSplit outperform previous benchmarks.

The dataset contains 60 times more split examples than prior datasets.

The best model scores 32 BLEU points higher on WebSplit.

Abstract

Split and rephrase is the task of breaking down a sentence into shorter ones that together convey the same meaning. We extract a rich new dataset for this task by mining Wikipedia's edit history: WikiSplit contains one million naturally occurring sentence rewrites, providing sixty times more distinct split examples and a ninety times larger vocabulary than the WebSplit corpus introduced by Narayan et al. (2017) as a benchmark for this task. Incorporating WikiSplit as training data produces a model with qualitatively better predictions that score 32 BLEU points above the prior best result on the WebSplit benchmark.