Quantitative Measurements of the Influence of Participant Roles during Peer Review Meetings

TL;DR

This study quantifies how different participant roles influence peer review meetings, revealing patterns that suggest tailored review types could enhance meeting effectiveness.

Contribution

It introduces a protocol analysis approach to measure role influence in PRMs, proposing specialized review types based on role focus and activity patterns.

Findings

Review activities vary by focus on form or content.

Different roles exhibit distinct activity patterns.

Tailored review types may improve PRM effectiveness.

Abstract

Peer review meetings (PRMs) are formal meetings during which peers systematically analyze artifacts to improve their quality and report on non-conformities. This paper presents an approach based on protocol analysis for quantifying the influence of participant roles during PRMs. Three views are used to characterize the seven defined participant roles. The project view defines three roles supervisor, procedure expert and developer. The meeting view defines two roles: author and reviewer, and the task view defines the roles reflecting direct and indirect interest in the artifact under review. The analysis, based on log-linear modeling, shows that review activities have different patterns, depending on their focus: form or content. The influence of each role is analyzed with respect to this focus. Interpretation of the quantitative data leads to the suggestion that PRMs could be improved by creating three different types of reviews, each of which collects together specific roles: form review, cognitive synchronization review and content review.