SEMAG: Self-Evolutionary Multi-Agent Code Generation

TL;DR

SEMAG introduces a self-evolving multi-agent framework for code generation that dynamically adapts workflows and models, achieving state-of-the-art accuracy and outperforming prior methods on benchmarks.

Contribution

It presents a novel framework that mimics human coding stages, enabling real-time model upgrades and adaptive workflows for improved code generation performance.

Findings

SEMAG achieves new state-of-the-art Pass@1 accuracy.

Outperforms prior methods by 3.3% on CodeContests.

Automatically identifies optimal models for enhanced performance.

Abstract

Large Language Models (LLMs) have made significant progress in handling complex programming tasks. However, current methods rely on manual model selection and fixed workflows, which limit their ability to adapt to changing task complexities. To address this, we propose SEMAG, a Self-Evolutionary Multi-Agent code Generation framework that mimics human coding practices. It decomposes programming tasks into stages, including planning, coding, debugging, and discussion, while adapting workflows to task difficulty. Its self-evolutionary agents can access the latest models in real time and automatically upgrade the backbone model. SEMAG sets new state-of-the-art Pass@1 accuracy across benchmarks. Using identical backbone models, SEMAG outperforms prior methods by 3.3% on CodeContests. When augmented with self-evolutionary model selection that automatically identifies optimal backbones, SEMAG reaches 52.6%, showcasing both framework effectiveness and adaptability to evolving LLM capabilities.