Content based Weighted Consensus Summarization

TL;DR

This paper introduces a content-based weighted consensus approach for multi-document summarization, leveraging pseudo-relevant summaries to improve ensemble performance and outperform existing methods on DUC datasets.

Contribution

It proposes a novel content-based weighted consensus method that accounts for individual system performance and content, enhancing ensemble summarization quality.

Findings

Outperforms existing consensus methods on DUC datasets

Effectively estimates system performance using pseudo-relevant summaries

Improves summary quality by integrating content and performance weights

Abstract

Multi-document summarization has received a great deal of attention in the past couple of decades. Several approaches have been proposed, many of which perform equally well and it is becoming in- creasingly difficult to choose one particular system over another. An ensemble of such systems that is able to leverage the strengths of each individual systems can build a better and more robust summary. Despite this, few attempts have been made in this direction. In this paper, we describe a category of ensemble systems which use consensus between the candidate systems to build a better meta-summary. We highlight two major shortcomings of such systems: the inability to take into account relative performance of individual systems and overlooking content of candidate summaries in favour of the sentence rankings. We propose an alternate method, content-based weighted consensus summarization, which address these concerns. We use pseudo-relevant summaries to estimate the performance of individual candidate systems, and then use this information to generate a better aggregate ranking. Experiments on DUC 2003 and DUC 2004 datasets show that the proposed system outperforms existing consensus-based techniques by a large margin.