GLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviews

TL;DR

The paper presents \\sys, a summarization method that extracts both common and unique opinions from scholarly reviews to assist area chairs in efficiently understanding review arguments, balancing comprehensiveness and conciseness.

Contribution

It introduces a novel summarization approach using Rational Speech Act-based uniqueness scores to better capture diverse opinions in scholarly reviews, improving review comprehension.

Findings

curately summarizes reviews with human-preferred discriminative quality.

Achieves comparable automatic metric performance to baseline methods.

Enhances review understanding for conference decision-making.

Abstract

Scientific peer review is essential for the quality of academic publications. However, the increasing number of paper submissions to conferences has strained the reviewing process. This surge poses a burden on area chairs who have to carefully read an ever-growing volume of reviews and discern each reviewer's main arguments as part of their decision process. In this paper, we introduce \sys, a summarization method designed to offer a concise yet comprehensive overview of scholarly reviews. Unlike traditional consensus-based methods, \sys extracts both common and unique opinions from the reviews. We introduce novel uniqueness scores based on the Rational Speech Act framework to identify relevant sentences in the reviews. Our method aims to provide a pragmatic glimpse into all reviews, offering a balanced perspective on their opinions. Our experimental results with both automatic metrics and human evaluation show that \sys generates more discriminative summaries than baseline methods in terms of human evaluation while achieving comparable performance with these methods in terms of automatic metrics.