Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair

TL;DR

This paper explores a novel approach in agentic automated program repair that co-generates bug reproduction tests and fixes within a single patch, improving efficiency and confidence in generated patches.

Contribution

It introduces a cogeneration framework for APR that produces both fixes and BRTs simultaneously, reducing the need for separate generation pipelines.

Findings

Cogeneration achieves comparable BRT generation rates to dedicated BRT agents.

Patch selectors using test change info improve plausibility of generated patches.

Analysis of cogeneration reveals root causes of failure modes.

Abstract

Bug Reproduction Tests (BRTs) have been used in many Automated Program Repair (APR) systems, primarily for validating promising fixes and aiding fix generation. In practice, when developers submit a patch, they often implement the BRT alongside the fix. Our experience deploying agentic APR reveals that developers similarly desire a BRT within AI-generated patches to increase their confidence. However, canonical APR systems tend to generate BRTs and fixes separately, and focus on producing only the fix in the final patch. In this paper, we study agentic APR in the context of cogeneration, where the APR agent is instructed to generate both a fix and a BRT in the same patch. We evaluate the effectiveness of different cogeneration strategies on 120 human-reported bugs at Google and characterize different cogeneration strategies by their influence on APR agent behavior. We develop and evaluate patch selectors that account for test change information to select patches with plausible fixes (and plausible BRTs). Finally, we analyze the root causes of failed cogeneration trajectories. Importantly, we show that cogeneration allows the APR agent to generate BRTs for at least as many bugs as a dedicated BRT agent, without compromising the generation rate of plausible fixes, thereby reducing engineering effort in maintaining and coordinating separate generation pipelines for fix and BRT at scale.