On The Impact of Merge Request Deviations on Code Review Practices

TL;DR

This paper investigates how deviations in merge request workflows affect code review analytics and ML models, proposing a detection method and demonstrating the importance of accounting for deviations to improve review process insights.

Contribution

It introduces a taxonomy of MR deviations, a few-shot learning detection method, and empirically shows their impact on review analytics and ML model performance.

Findings

Deviations occur in 37% of MRs.

Detection method achieves 91% accuracy.

Excluding deviations improves ML review time predictions by up to 2.25x.

Abstract

Code review is a key practice in software engineering, ensuring quality and collaboration. However, industrial Merge Request (MR) workflows often deviate from standardized review processes, with many MRs serving non-review purposes (e.g., drafts, rebases, or dependency updates). We term these cases deviations and hypothesize that ignoring them biases analytics and undermines ML models for review analysis. We identify seven deviation categories, occurring in 37.02% of MRs, and propose a few-shot learning detection method (91% accuracy). By excluding deviations, ML models predicting review completion time improve performance in 53.33% of cases (up to 2.25x) and exhibit significant shifts in feature importance (47% overall, 60% top-*k*). Our contributions include: (1) a taxonomy of MR deviations, (2) an AI-driven detection approach, and (3) empirical evidence of their impact on ML-based review analytics. This work aids practitioners in optimizing review efforts and ensuring reliable insights.