Repair Ingredients Are All You Need: Improving Large Language Model-Based Program Repair via Repair Ingredients Search

TL;DR

ReinFix enhances large language model-based program repair by integrating static analysis and historical bug data to improve patch accuracy and success rates on standard benchmarks.

Contribution

This paper introduces ReinFix, a framework that enables LLMs to search for repair ingredients during reasoning and solution phases, improving program repair effectiveness.

Findings

ReinFix fixes 146 bugs on Defects4J V1.2, 32 more than baselines.

ReinFix fixes 38 more bugs than SOTA on Defects4J V2.0.

ReinFix maintains top performance on recent, leakage-free benchmarks.

Abstract

Automated Program Repair (APR) techniques aim to automatically fix buggy programs. Among these, Large Language Model-based (LLM-based) approaches have shown great promise. Recent advances demonstrate that directly leveraging LLMs can achieve leading results. However, these techniques remain suboptimal in generating contextually relevant and accurate patches, as they often overlook repair ingredients crucial for practical program repair. In this paper, we propose ReinFix, a novel framework that enables LLMs to autonomously search for repair ingredients throughout both the reasoning and solution phases of bug fixing. In the reasoning phase, ReinFix integrates static analysis tools to retrieve internal ingredients, such as variable definitions, to assist the LLM in root cause analysis when it encounters difficulty understanding the context. During the solution phase, when the LLM lacks experience in fixing specific bugs, ReinFix searches for external ingredients from historical bug fixes with similar bug patterns, leveraging both the buggy code and its root cause to guide the LLM in identifying appropriate repair actions, thereby increasing the likelihood of generating correct patches. Evaluations on two popular benchmarks (Defects4J V1.2 and V2.0) demonstrate the effectiveness of our approach over SOTA baselines. Notably, ReinFix fixes 146 bugs, which is 32 more than the baselines on Defects4J V1.2. On Defects4J V2.0, ReinFix fixes 38 more bugs than the SOTA. Importantly, when evaluating on the recent benchmarks that are free of data leakage risk, ReinFix also maintains the best performance.