EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configuration

TL;DR

EvoConfig is a self-evolving multi-agent framework that enhances environment configuration efficiency and accuracy for large language models by integrating expert diagnosis and dynamic feedback, significantly improving success rates and debugging capabilities.

Contribution

The paper introduces EvoConfig, a novel self-evolving multi-agent system with expert diagnosis and feedback mechanisms for improved environment configuration.

Findings

EvoConfig matches state-of-the-art on Repo2Run datasets.

Achieves 78.1% success rate on Envbench, outperforming previous methods.

Demonstrates superior debugging and error correction capabilities.

Abstract

A reliable executable environment is the foundation for ensuring that large language models solve software engineering tasks. Due to the complex and tedious construction process, large-scale configuration is relatively inefficient. However, most methods always overlook fine-grained analysis of the actions performed by the agent, making it difficult to handle complex errors and resulting in configuration failures. To address this bottleneck, we propose EvoConfig, an efficient environment configuration framework that optimizes multi-agent collaboration to build correct runtime environments. EvoConfig features an expert diagnosis module for fine-grained post-execution analysis, and a self-evolving mechanism that lets expert agents self-feedback and dynamically adjust error-fixing priorities in real time. Empirically, EvoConfig matches the previous state-of-the-art Repo2Run on Repo2Run's 420 repositories, while delivering clear gains on harder cases: on the more challenging Envbench, EvoConfig achieves a 78.1% success rate, outperforming Repo2Run by 7.1%. Beyond end-to-end success, EvoConfig also demonstrates stronger debugging competence, achieving higher accuracy in error identification and producing more effective repair recommendations than existing methods.