Extractive Summarization: Limits, Compression, Generalized Model and Heuristics

TL;DR

This paper investigates the theoretical limits of extractive summarization, introduces a generalized model encompassing various summarization dimensions, and compares multiple algorithms against state-of-the-art methods on DUC datasets.

Contribution

It establishes empirical limits on extractive summarization performance, proposes a comprehensive generalized model, and evaluates algorithms within a unified framework.

Findings

Empirical recall and F1-score limits on DUC datasets.

A new generalized model integrating multiple summarization dimensions.

Comparative analysis of algorithms against state-of-the-art methods.

Abstract

Due to its promise to alleviate information overload, text summarization has attracted the attention of many researchers. However, it has remained a serious challenge. Here, we first prove empirical limits on the recall (and F1-scores) of extractive summarizers on the DUC datasets under ROUGE evaluation for both the single-document and multi-document summarization tasks. Next we define the concept of compressibility of a document and present a new model of summarization, which generalizes existing models in the literature and integrates several dimensions of the summarization, viz., abstractive versus extractive, single versus multi-document, and syntactic versus semantic. Finally, we examine some new and existing single-document summarization algorithms in a single framework and compare with state of the art summarizers on DUC data.