Early-Stage Prediction of Review Effort in AI-Generated Pull Requests

TL;DR

This paper presents a model to predict high-maintenance AI-generated pull requests early, enabling efficient triage by identifying complex contributions before human review, thus reducing effort and improving workflow.

Contribution

Introduces a creation-time Circuit Breaker model using static complexity cues to identify high-effort AI-generated pull requests before review.

Findings

Model achieves AUC 0.96 in predicting high-maintenance PRs

At 20% review budget, captures 69% of high-effort PRs

Enables fast-fail of costly, low-quality contributions

Abstract

As AI coding agents evolve from autocomplete tools to autonomous "AI workforce" teammates, they introduce a critical new bottleneck: human maintainers must now manage complex interaction loops rather than just reviewing code. Analyzing 33,707 agent-authored PRs, we uncover a stark two-regime reality: agents excel at narrow automation (28.3% of PRs merge instantly), but frequently fail at iterative refinement, leading to "ghosting" (abandonment) when faced with subjective feedback. This creates a hidden "attention tax" on maintainers. We introduce a creation-time Circuit Breaker model to predict high-maintenance PRs before human review begins. By leveraging simple static complexity cues (e.g., file types, patch size), our model identifies the "expensive tail" of contributions with AUC 0.96, enabling a gated triage process. At a 20% review budget, this approach captures 69% of the high-effort PRs, effectively allowing maintainers to fast-fail costly, low-quality agent contributions while fast-tracking simple fixes.