Solving reviewer assignment problem in software peer review: An approach based on preference matrix and asymmetric TSP model

TL;DR

This paper presents an optimization approach for reviewer assignment in software peer review using preference matrices and an asymmetric TSP model, improving assignment quality over random strategies.

Contribution

It introduces a novel method combining preference matrices and ATSP for reviewer assignment, validated through comparative experiments.

Findings

Preference-based assignments outperform random assignment.

Merged preference matrices yield only slight improvements over random.

Students prefer harmonious reviewer-author pairings despite high-level students' reluctance.

Abstract

Optimized reviewer assignment can effectively utilize limited intellectual resources and significantly assure review quality in various scenarios such as paper selection in conference or journal, proposal selection in funding agencies and so on. However, little research on reviewer assignment of software peer review has been found. In this study, an optimization approach is proposed based on students' preference matrix and the model of asymmetric traveling salesman problem (ATSP). Due to the most critical role of rule matrix in this approach, we conduct a questionnaire to obtain students' preference matrices and convert them to rule matrices. With the help of software ILOG CPLEX, the approach is accomplished by controlling the exit criterion of ATSP model. The comparative study shows that the assignment strategies with both reviewers' preference matrix and authors' preference matrix get better performance than the random assignment. Especially, it is found that the performance is just a little better than that of random assignment when the reviewers' and authors' preference matrices are merged. In other words, the majority of students have a strong wish of harmonious development even though high-level students are not willing to do that.