Software Self-Extension with SelfEvolve: an Agentic Architecture for Runtime Code Generation

TL;DR

SelfEvolve introduces an agentic architecture that enables software systems to autonomously generate and integrate new functionalities at runtime, demonstrating significant improvements over existing code generation baselines.

Contribution

The paper presents SelfEvolve, a novel architecture for runtime self-extension of software capabilities using autonomous code generation with large language models.

Findings

Achieved 92.7% Pass@1 on 11 self-extension tasks.

Outperformed baselines like AutoGen, MetaGPT, and AgentCoder.

61.8% improvement over the best baseline, Autogen.

Abstract

Traditional self-adaptive systems automatically reconfigure existing components in response to changing requirements, but provide limited support for the generation of novel functionalities. The software generation capabilities of large language models (LLMs) open the possibility to create entirely new modules at runtime, enabling a form of self-evolution beyond traditional self-adaptation. We present SelfEvolve, an orchestrated agentic pipeline architecture enabling runtime self-extension--the autonomous addition of new capabilities during execution--as a preliminary form of self-evolution. Self-extension focuses on the autonomous generation and integration of new functions, based on user requests, without requiring a system restart or developer intervention. Evaluation of our architecture across 11 self-extension tasks demonstrates an average Pass@1 of 92.7% (51/55), outperforming developer-focused code generation baselines like AutoGen, MetaGPT, and AgentCoder. SelfEvolve achieves 61.8% improvement over the best baseline, i.e. Autogen, with statistical significance. This work demonstrates the feasibility of runtime capability extension through autonomous code generation. This provides preliminary evidence for a paradigm in which systems autonomously evolve to satisfy user needs, paving the way towards individualised, self-improving systems.