Generating Summaries for Scientific Paper Review

TL;DR

This paper investigates the use of neural language models to automatically generate review summaries for scientific papers, aiming to reduce reviewer burden and improve review quality.

Contribution

It introduces a new dataset of scientific papers and reviews, evaluates neural summarization models on this task, and discusses future research directions.

Findings

Neural models show potential for review summary generation

A new dataset from NeurIPS papers and reviews is released

Initial results demonstrate feasibility of automatic review summaries

Abstract

The review process is essential to ensure the quality of publications. Recently, the increase of submissions for top venues in machine learning and NLP has caused a problem of excessive burden on reviewers and has often caused concerns regarding how this may not only overload reviewers, but also may affect the quality of the reviews. An automatic system for assisting with the reviewing process could be a solution for ameliorating the problem. In this paper, we explore automatic review summary generation for scientific papers. We posit that neural language models have the potential to be valuable candidates for this task. In order to test this hypothesis, we release a new dataset of scientific papers and their reviews, collected from papers published in the NeurIPS conference from 2013 to 2020. We evaluate state of the art neural summarization models, present initial results on the feasibility of automatic review summary generation, and propose directions for the future.