BanditSum: Extractive Summarization as a Contextual Bandit

TL;DR

BanditSum introduces a reinforcement learning approach to extractive summarization, framing it as a contextual bandit problem, leading to improved ROUGE scores and faster convergence compared to existing methods.

Contribution

It is the first to treat extractive summarization as a contextual bandit problem and trains neural networks without heuristic labels.

Findings

Achieves state-of-the-art or comparable ROUGE scores.

Converges faster than competing methods.

Performs better when key sentences are late in the document.

Abstract

In this work, we propose a novel method for training neural networks to perform single-document extractive summarization without heuristically-generated extractive labels. We call our approach BanditSum as it treats extractive summarization as a contextual bandit (CB) problem, where the model receives a document to summarize (the context), and chooses a sequence of sentences to include in the summary (the action). A policy gradient reinforcement learning algorithm is used to train the model to select sequences of sentences that maximize ROUGE score. We perform a series of experiments demonstrating that BanditSum is able to achieve ROUGE scores that are better than or comparable to the state-of-the-art for extractive summarization, and converges using significantly fewer update steps than competing approaches. In addition, we show empirically that BanditSum performs significantly better than competing approaches when good summary sentences appear late in the source document.