Beyond Localization: Recoverable Headroom and Residual Frontier in Repository-Level RAG-APR

TL;DR

This paper investigates the limits of repository-level automated program repair (APR) after strengthening localization, identifying residual gains and the factors influencing repair success across different paradigms.

Contribution

It introduces a protocol to analyze recoverable gains in repository-level RAG-APR and evaluates the impact of localization, context, and interface design on repair outcomes.

Findings

Oracle Localization improves repair success but remains below 50%.

Additional context probes yield diminishing returns after initial gains.

Prompt-level fusion offers limited improvements beyond existing methods.

Abstract

Repository-level automated program repair (APR) increasingly treats stronger localization as the main path to better repair. We ask a more targeted question: once localization is strengthened, which post-localization levers still provide recoverable gains, which are bounded within our protocol, and what residual frontier remains? We study this question on SWE-bench Lite with three representative repository-level RAG-APR paradigms, Agentless, KGCompass, and ExpeRepair. Our protocol combines Oracle Localization, within-pool Best-of-K, fixed-interface added context probes with per-condition same-token filler controls and same-repository hard negatives, and a common-wrapper oracle check. Oracle Localization improves all three systems, but Oracle success still stays below 50%. Extra candidate diversity still helps inside the sampled 10-patch pools, but that headroom saturates quickly. Under the two fixed interfaces, most informative added context conditions still outperform their own matched controls. The common-wrapper check shows different system responses: under a common wrapper, gains remain large for KGCompass and ExpeRepair, while Agentless changes more with builder choice. Prompt-level fusion still leaves a large residual frontier: the best fixed probe adds only 6 solved instances beyond the native three-system Solved@10 union. Overall, stronger localization, bounded search, evidence quality, and interface design all shape repository-level repair outcomes.