Abstractive Document Summarization without Parallel Data

TL;DR

This paper introduces an unsupervised abstractive summarization method that generates summaries without needing parallel article-summary data, using only collections of summaries and articles, and performs well on benchmarks and real-world tasks.

Contribution

The authors propose a novel unsupervised approach combining sentence extraction and paraphrasing trained on pseudo-data, eliminating the need for paired datasets.

Findings

Competitive performance on CNN/DailyMail benchmark

Effective in generating press releases from scientific articles

Outperforms some supervised methods in low-resource settings

Abstract

Abstractive summarization typically relies on large collections of paired articles and summaries. However, in many cases, parallel data is scarce and costly to obtain. We develop an abstractive summarization system that relies only on large collections of example summaries and non-matching articles. Our approach consists of an unsupervised sentence extractor that selects salient sentences to include in the final summary, as well as a sentence abstractor that is trained on pseudo-parallel and synthetic data, that paraphrases each of the extracted sentences. We perform an extensive evaluation of our method: on the CNN/DailyMail benchmark, on which we compare our approach to fully supervised baselines, as well as on the novel task of automatically generating a press release from a scientific journal article, which is well suited for our system. We show promising performance on both tasks, without relying on any article-summary pairs.