Learning to Rank Salient Content for Query-focused Summarization

TL;DR

This paper introduces a Learning-to-Rank integrated approach for query-focused summarization that improves relevance and coherence of summaries, outperforming state-of-the-art models on key benchmarks with lower training costs.

Contribution

The study presents a novel shared decoder architecture combining LTR with summarization, enhancing summary relevance and coherence in query-focused tasks.

Findings

Outperforms on QMSum benchmark in all metrics

Matches state-of-the-art on SQuALITY in two metrics

Achieves higher Rouge-L and BertScore scores, indicating better relevance

Abstract

This study examines the potential of integrating Learning-to-Rank (LTR) with Query-focused Summarization (QFS) to enhance the summary relevance via content prioritization. Using a shared secondary decoder with the summarization decoder, we carry out the LTR task at the segment level. Compared to the state-of-the-art, our model outperforms on QMSum benchmark (all metrics) and matches on SQuALITY benchmark (2 metrics) as measured by Rouge and BertScore while offering a lower training overhead. Specifically, on the QMSum benchmark, our proposed system achieves improvements, particularly in Rouge-L (+0.42) and BertScore (+0.34), indicating enhanced understanding and relevance. While facing minor challenges in Rouge-1 and Rouge-2 scores on the SQuALITY benchmark, the model significantly excels in Rouge-L (+1.47), underscoring its capability to generate coherent summaries. Human evaluations emphasize the efficacy of our method in terms of relevance and faithfulness of the generated summaries, without sacrificing fluency. A deeper analysis reveals our model's superiority over the state-of-the-art for broad queries, as opposed to specific ones, from a qualitative standpoint. We further present an error analysis of our model, pinpointing challenges faced and suggesting potential directions for future research in this field.