Exploring Neural Models for Query-Focused Summarization

TL;DR

This paper systematically explores neural models for query-focused summarization, comparing extractive-abstractive and end-to-end approaches, and introduces extensions that achieve state-of-the-art results on the QMSum dataset.

Contribution

It provides a comprehensive analysis of neural methods for QFS, introduces novel modeling extensions, and demonstrates improved performance with transfer learning strategies.

Findings

State-of-the-art ROUGE scores achieved on QMSum dataset

Models produce more comprehensive summaries in human evaluations

Transfer learning significantly boosts model performance

Abstract

Query-focused summarization (QFS) aims to produce summaries that answer particular questions of interest, enabling greater user control and personalization. While recently released datasets, such as QMSum or AQuaMuSe, facilitate research efforts in QFS, the field lacks a comprehensive study of the broad space of applicable modeling methods. In this paper we conduct a systematic exploration of neural approaches to QFS, considering two general classes of methods: two-stage extractive-abstractive solutions and end-to-end models. Within those categories, we investigate existing models and explore strategies for transfer learning. We also present two modeling extensions that achieve state-of-the-art performance on the QMSum dataset, up to a margin of 3.38 ROUGE-1, 3.72 ROUGE2, and 3.28 ROUGE-L when combined with transfer learning strategies. Results from human evaluation suggest that the best models produce more comprehensive and factually consistent summaries compared to a baseline model. Code and checkpoints are made publicly available: https://github.com/salesforce/query-focused-sum.