Dispute Resolution in Peer Review with Abstract Argumentation and OWL DL

TL;DR

This paper proposes a formal argumentation-based method using OWL DL to improve transparency and fairness in peer review dispute resolution, addressing biases and increasing process rigor.

Contribution

It introduces a novel formalization of peer review disputes with abstract argumentation frameworks implemented in OWL DL, enabling systematic resolution.

Findings

Frameworks are well-founded and decidable in linear time

Annotated peer reviews demonstrate the method's applicability

Potential to enhance review fairness and transparency

Abstract

The peer review process for scientific publications faces significant challenges due to the increasing volume of submissions and inherent reviewer biases. While artificial intelligence offers the potential to facilitate the process, it also risks perpetuating biases present in training data. This research addresses these challenges by applying formal methods from argumentation theory to support transparent and unbiased dispute resolution in peer review. Specifically, we conceptualize scientific peer review as a single mixed argumentative dispute between manuscript authors and reviewers and formalize it using abstract argumentation frameworks. We analyze the resulting peer review argumentation frameworks from semantic, graph-theoretic, and computational perspectives, showing that they are well-founded and decidable in linear time. These frameworks are then implemented using OWL DL and resolved with reasoning engines. We validate our approach by annotating a corpus of scientific peer reviews with abstract argumentation frameworks and applying a proof of concept to resolve the annotated disputes. The results demonstrate that integrating our method could enhance the quality of published work by providing a more rigorous and systematic approach to accounting reviewer arguments.