RevASIDE: Assignment of Suitable Reviewer Sets for Publications from Fixed Candidate Pools

TL;DR

RevASIDE is an automated system for assigning suitable reviewer sets from fixed candidate pools, considering expertise, authority, diversity, and interest, improving accuracy over baseline methods.

Contribution

It introduces RevASIDE, a novel reviewer assignment system that does not require manual reviewer profiles and considers multiple suitability factors.

Findings

RevASIDE outperforms baseline methods in reviewer set assignment accuracy.

Document embeddings outperform simple text similarity in expert search.

New datasets for expert search and reviewer assignment tasks are provided.

Abstract

Scientific publishing heavily relies on the assessment of quality of submitted manuscripts by peer reviewers. Assigning a set of matching reviewers to a submission is a highly complex task which can be performed only by domain experts. We introduce RevASIDE, a reviewer recommendation system that assigns suitable sets of complementing reviewers from a predefined candidate pool without requiring manually defined reviewer profiles. Here, suitability includes not only reviewers' expertise, but also their authority in the target domain, their diversity in their areas of expertise and experience, and their interest in the topics of the manuscript. We present three new data sets for the expert search and reviewer set assignment tasks and compare the usefulness of simple text similarity methods to document embeddings for expert search. Furthermore, an quantitative evaluation demonstrates significantly better results in reviewer set assignment compared to baselines. A qualitative evaluation also shows their superior perceived quality.