Memory-Efficient Large Language Models for Program Repair with Semantic-Guided Patch Generation

TL;DR

This paper introduces FLAMES, a memory-efficient large language model-based program repair method that uses semantic-guided patch generation and greedy decoding to improve repair effectiveness while significantly reducing memory usage.

Contribution

FLAMES is a novel approach that combines semantic-guided search with greedy decoding to enhance memory efficiency and repair accuracy in LLM-based program repair.

Findings

Reduces memory consumption by up to 83% compared to baseline methods.

Fixes 10 more bugs on Defects4J than the best existing baseline.

Achieves 12% and 36.5% improvements in correct patches on HumanEval-Java and TransformedD4J datasets.

Abstract

In this paper, we first show that increases in beam size, even for small-sized LLMs (1B-7B params), require extensive GPU usage, leading to up to 80% of recurring crashes due to memory overloads in LLM-based APR. Seemingly simple solutions to reduce memory consumption are (1) to quantize LLM models, i.e., converting the weights of an LLM from high-precision values to lower-precision ones, and (2) to make beam search sequential, i.e., forwarding each beam through the model sequentially and then concatenating them back into a single output. However, we show that these approaches still do not work via both theoretical analysis and experiments. To address this, we introduce FLAMES, a novel LLM-based APR technique that employs semantic-guided patch generation to enhance repair effectiveness and memory efficiency. Unlike conventional methods that rely on beam search, FLAMES utilizes greedy decoding to enhance memory efficiency while steering the search towards more potentially good repair candidates via a semantic-guided best-first search algorithm. At each decoding step, FLAMES uses semantic feedback from test validation, such as the number of passing and failing test cases, to select the most promising token to explore further. Our empirical evaluation on Defects4J shows thatFLAMES substantially reduces memory consumption by up to 83% compared to LLM-based APR without compromising time efficiency. Moreover, FLAMES correctly fixes 133 bugs on Defects4J, fixing 10 bugs more than the best baseline. Additionally, these improvements also generalize to the HumanEval-Java and TransformedD4J datasets, where FLAMES generates 12% and 36.5% more correct patches, respectively, than the best baseline.