TextBFGS: A Case-Based Reasoning Approach to Code Optimization via Error-Operator Retrieval

TL;DR

TextBFGS introduces a case-based reasoning framework for code optimization with LLMs, leveraging historical correction trajectories to improve efficiency and effectiveness over traditional trial-and-error methods.

Contribution

It presents a novel CBR approach inspired by Quasi-Newton methods, maintaining a dynamic case base to enhance iterative code correction in LLMs.

Findings

Outperforms stateless baselines in Python code tasks

Achieves higher pass rates with fewer model calls

Demonstrates effective self-evolving correction system

Abstract

Iterative code generation with Large Language Models (LLMs) can be viewed as an optimization process guided by textual feedback. However, existing LLM self-correction methods predominantly operate in a stateless, trial-and-error manner akin to first-order search, failing to leverage past problem-solving experiences. To bridge this gap, we introduce TextBFGS, a Case-Based Reasoning (CBR) framework inspired by the Quasi-Newton optimization method. Instead of retrieving raw, unstructured textual instances, TextBFGS maintains a dynamic Case Base of historical "Error-to-Operator" correction trajectories to approximate the semantic curvature (inverse Hessian matrix) of the task. Specifically, given a textual error feedback (the target problem), TextBFGS retrieves analogous historical correction patterns (Retrieve) and applies these abstract operators to refine the current code (Reuse/Revise). Furthermore, successful adaptations are continuously retained back into the Case Base (Retain), enabling a self-evolving system. Empirical evaluations on Python code optimization tasks (HumanEval, MBPP) demonstrate that TextBFGS significantly outperforms stateless baselines. It achieves superior pass rates with fewer model calls, establishing an efficient, experience-driven paradigm for LLM-based code optimization.