Input Reduction Enhanced LLM-based Program Repair

TL;DR

This paper introduces ReduceFix, a method that automatically reduces large failure-inducing test inputs to improve the effectiveness of LLM-based automated program repair, demonstrating significant performance gains on a new long-input benchmark.

Contribution

The paper proposes ReduceFix, a novel input reduction approach that enhances LLM-based program repair by minimizing test inputs without human effort, and introduces LFTBench, a benchmark for long-input APR with real bugs.

Findings

ReduceFix shrinks inputs by 89.1% on average.

Improves pass@10 by up to 53.8% with input reduction.

Enhances repair success of ChatRepair and CREF by 21.3% and 2.6%.

Abstract

Large Language Models (LLMs) have shown great potential in Automated Program Repair (APR). Test inputs, being crucial for reasoning the root cause of failures, are always included in the prompt for LLM-based APR. Unfortunately, LLMs struggle to retain key information in long prompts. When the test inputs are extensive in the prompt, this may trigger the "lost-in-the-middle" issue, compromising repair performance. ReduceFix prompts an LLM to generate a reducer that minimizes failure-inducing test inputs without human effort, and then feeds the reduced failure-inducing inputs to guide patch generation. For targeted evaluation, we constructed LFTBench, the first long-input APR benchmark with 200 real bugs from 20 programming tasks, each paired with a failure-inducing input whose median size is 1 MB. On this benchmark, ReduceFix shrinks inputs by 89.1% on average and improves overall pass@10 by up to 53.8% relative to a prompt that includes the original test, and by 17.6% compared with omitting the test entirely. Adding the same reduction step to ChatRepair and CREF increases their fix rate by 21.3% and 2.6%, respectively, without other changes. Our gains hold against a ddmin-only reducing template baseline and transfer to repository-level OSS-Fuzz cases. Ablation studies further highlight the impact of input length and compressed failure information on repair success. These results underscore that automatically reducing failing inputs is a practical and powerful complement to LLM-based APR, significantly improving its scalability and effectiveness.