A Multi-Document Coverage Reward for RELAXed Multi-Document Summarization

TL;DR

This paper introduces a reinforcement learning approach using a multi-document coverage reward with RELAX to improve multi-document summarization, resulting in higher ROUGE and METEOR scores and better input document coverage.

Contribution

It proposes a novel reward function combining reference-based metrics and input coverage, fine-tuned with RELAX for efficient multi-document summarization.

Findings

Significant ROUGE score improvements (+0.95 pp)

Enhanced input document coverage across all documents

Competitive results with existing literature

Abstract

Multi-document summarization (MDS) has made significant progress in recent years, in part facilitated by the availability of new, dedicated datasets and capacious language models. However, a standing limitation of these models is that they are trained against limited references and with plain maximum-likelihood objectives. As for many other generative tasks, reinforcement learning (RL) offers the potential to improve the training of MDS models; yet, it requires a carefully-designed reward that can ensure appropriate leverage of both the reference summaries and the input documents. For this reason, in this paper we propose fine-tuning an MDS baseline with a reward that balances a reference-based metric such as ROUGE with coverage of the input documents. To implement the approach, we utilize RELAX (Grathwohl et al., 2018), a contemporary gradient estimator which is both low-variance and unbiased, and we fine-tune the baseline in a few-shot style for both stability and computational efficiency. Experimental results over the Multi-News and WCEP MDS datasets show significant improvements of up to +0.95 pp average ROUGE score and +3.17 pp METEOR score over the baseline, and competitive results with the literature. In addition, they show that the coverage of the input documents is increased, and evenly across all documents.