Content Selection in Deep Learning Models of Summarization

TL;DR

This paper investigates how deep learning models perform content selection in summarization across various domains, revealing that simpler models often match complex ones and questioning the added value of deep learning for certain datasets.

Contribution

It demonstrates that sophisticated features do not outperform simpler models in summarization, suggesting easier domain adaptation and highlighting the need for new sentence representations or external knowledge.

Findings

Simpler models perform comparably to complex deep learning models.

Deep learning models may not be necessary for domains with large datasets like news.

New representations or external knowledge are needed for improved summarization.

Abstract

We carry out experiments with deep learning models of summarization across the domains of news, personal stories, meetings, and medical articles in order to understand how content selection is performed. We find that many sophisticated features of state of the art extractive summarizers do not improve performance over simpler models. These results suggest that it is easier to create a summarizer for a new domain than previous work suggests and bring into question the benefit of deep learning models for summarization for those domains that do have massive datasets (i.e., news). At the same time, they suggest important questions for new research in summarization; namely, new forms of sentence representations or external knowledge sources are needed that are better suited to the summarization task.