Generating Query-Focused Summarization Datasets from Query-Free Summarization Datasets

TL;DR

This paper introduces a method to generate evidence-based queries from query-free datasets to improve query-focused summarization, showing competitive results with original queries.

Contribution

It presents an evidence-based query generation model that enables the use of query-free datasets for QFS tasks, bridging a gap in dataset availability.

Findings

Generated queries are similar to original queries in QFS datasets.

Summaries from evidence-based queries achieve competitive ROUGE scores.

The approach supports effective query-focused summarization without query annotations.

Abstract

Large-scale datasets are widely used to perform summarization tasks, but they may not include queries alongside documents and summaries. In the search for suitable datasets for Query-Focused Summarization (QFS), we identify two research questions: Is it possible to automatically generate evidence-based query keywords from query-free datasets? Does evidence-based query generation support the QFS task? This paper proposes an evidence-based model to generate queries from query-free datasets. To evaluate our model intrinsically, we compare the similarity between the original queries and the system-generated queries of two QFS datasets. We also perform summarization tasks using different pre-trained models, as well as a state-of-the-art (SOTA) QFS model, to measure the extrinsic performance of our query generation approach. Experimental results indicate that summaries generated using evidence-based queries achieve competitive ROUGE scores compared to those generated from the original queries.