MinWikiSplit: A Sentence Splitting Corpus with Minimal Propositions

TL;DR

MinWikiSplit introduces a large corpus of 203,000 aligned complex and simplified sentence pairs, focusing on minimal propositions to aid in developing sentence splitting methods for improved text simplification and downstream NLP tasks.

Contribution

This paper presents a novel sentence splitting corpus emphasizing minimal propositions, filling a gap in existing datasets by providing detailed, aligned complex and simplified sentence pairs.

Findings

Corpus contains 203,000 aligned sentence pairs.

Each sentence is broken into minimal, self-contained propositions.

Facilitates development of sentence splitting methods for NLP applications.

Abstract

We compiled a new sentence splitting corpus that is composed of 203K pairs of aligned complex source and simplified target sentences. Contrary to previously proposed text simplification corpora, which contain only a small number of split examples, we present a dataset where each input sentence is broken down into a set of minimal propositions, i.e. a sequence of sound, self-contained utterances with each of them presenting a minimal semantic unit that cannot be further decomposed into meaningful propositions. This corpus is useful for developing sentence splitting approaches that learn how to transform sentences with a complex linguistic structure into a fine-grained representation of short sentences that present a simple and more regular structure which is easier to process for downstream applications and thus facilitates and improves their performance.