Iterative Experience Refinement of Software-Developing Agents

TL;DR

This paper introduces an iterative experience refinement framework for LLM-powered software agents, enabling them to adapt and improve their experiences during task execution, leading to better performance and efficiency.

Contribution

The paper proposes a novel iterative experience refinement approach with two patterns and experience elimination, enhancing adaptability and efficiency of LLM agents in software development tasks.

Findings

Successive pattern yields higher performance but less stability.

Cumulative pattern offers more stable results.

Experience elimination reduces required experience data to 11.54%.

Abstract

Autonomous agents powered by large language models (LLMs) show significant potential for achieving high autonomy in various scenarios such as software development. Recent research has shown that LLM agents can leverage past experiences to reduce errors and enhance efficiency. However, the static experience paradigm, reliant on a fixed collection of past experiences acquired heuristically, lacks iterative refinement and thus hampers agents' adaptability. In this paper, we introduce the Iterative Experience Refinement framework, enabling LLM agents to refine experiences iteratively during task execution. We propose two fundamental patterns: the successive pattern, refining based on nearest experiences within a task batch, and the cumulative pattern, acquiring experiences across all previous task batches. Augmented with our heuristic experience elimination, the method prioritizes high-quality and frequently-used experiences, effectively managing the experience space and enhancing efficiency. Extensive experiments show that while the successive pattern may yield superior results, the cumulative pattern provides more stable performance. Moreover, experience elimination facilitates achieving better performance using just 11.54% of a high-quality subset.