LLM Based Multi-Document Summarization Exploiting Main-Event Biased Monotone Submodular Content Extraction

TL;DR

This paper introduces a novel extract-rewrite method for multi-document news summarization that emphasizes the main event using a main-event biased monotone submodular function, improving objectivity and coherence.

Contribution

It proposes a main-event biased monotone submodular content selection combined with LLM-based rewriting, enhancing objectivity and coherence in multi-document summarization.

Findings

Outperforms baseline methods in content coverage and coherence.

Achieves higher scores on objective metrics and human evaluations.

Effectively captures the main event with improved informativeness.

Abstract

Multi-document summarization is a challenging task due to its inherent subjective bias, highlighted by the low inter-annotator ROUGE-1 score of 0.4 among DUC-2004 reference summaries. In this work, we aim to enhance the objectivity of news summarization by focusing on the main event of a group of related news documents and presenting it coherently with sufficient context. Our primary objective is to succinctly report the main event, ensuring that the summary remains objective and informative. To achieve this, we employ an extract-rewrite approach that incorporates a main-event biased monotone-submodular function for content selection. This enables us to extract the most crucial information related to the main event from the document cluster. To ensure coherence, we utilize a fine-tuned Language Model (LLM) for rewriting the extracted content into a coherent text. The evaluation using objective metrics and human evaluators confirms the effectiveness of our approach, as it surpasses potential baselines, demonstrating excellence in both content coverage, coherence, and informativeness.