BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle
Peter West, Ari Holtzman, Jan Buys, Yejin Choi

TL;DR
This paper introduces BottleSum, an unsupervised and self-supervised sentence summarization method based on the Information Bottleneck principle, leveraging pretrained language models to produce concise summaries without direct supervision.
Contribution
It presents a novel application of the Information Bottleneck principle to unsupervised and self-supervised sentence summarization using pretrained language models.
Findings
Extractive summarization outperforms other unsupervised models on multiple metrics.
Self-supervised abstractive summaries outperform unsupervised baselines in human evaluations.
The approach is efficient and does not require labeled data.
Abstract
The principle of the Information Bottleneck (Tishby et al. 1999) is to produce a summary of information X optimized to predict some other relevant information Y. In this paper, we propose a novel approach to unsupervised sentence summarization by mapping the Information Bottleneck principle to a conditional language modelling objective: given a sentence, our approach seeks a compressed sentence that can best predict the next sentence. Our iterative algorithm under the Information Bottleneck objective searches gradually shorter subsequences of the given sentence while maximizing the probability of the next sentence conditioned on the summary. Using only pretrained language models with no direct supervision, our approach can efficiently perform extractive sentence summarization over a large corpus. Building on our unsupervised extractive summarization (BottleSumEx), we then present a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
