MAS-ZERO: Designing Multi-Agent Systems with Zero Supervision

TL;DR

MAS-ZERO introduces a self-evolving, inference-time framework for automatic multi-agent system design that adapts to each problem without validation sets, improving performance across various complex tasks.

Contribution

It is the first framework enabling dynamic, self-evolved MAS configuration at inference time without validation data, enhancing adaptability and performance.

Findings

Outperforms manual and automatic baselines in reasoning, coding, and agentic tasks.

Achieves up to 16.69% accuracy improvement in reasoning tasks.

Maintains cost efficiency while improving accuracy.

Abstract

Multi-agent systems (MAS) leveraging the impressive capabilities of Large Language Models (LLMs) hold significant potential for tackling complex tasks. However, most current MAS depend on manually designed agent roles and communication protocols. These manual designs often fail to align with the underlying LLMs' strengths and struggle to adapt to novel tasks. Recent automatic MAS approaches attempt to mitigate these limitations but typically necessitate a validation set for tuning and yield static MAS designs lacking adaptability during inference, while also removing the flexibility to reduce to simpler systems. We introduce MAS-ZERO, the first self-evolved, inference-time framework for automatic MAS design. MAS-ZERO employs meta-level design to iteratively design, critique, and refine MAS configurations tailored to each problem instance, without requiring a validation set. Critically, it enables dynamic problem decomposition and agent composition through meta-feedback on solvability and completeness, and reduction to simpler systems when appropriate. Experiments across reasoning (math and graduate-level QA), coding, and agentic (search-based) benchmarks, using both closed-source and open-source LLM backbones of varying sizes, demonstrate that MAS-ZERO outperforms strong manual and automatic MAS baselines. It achieves substantial average accuracy improvements of up to 16.69% on reasoning, 16.66% on coding, and 5.45% on agentic tasks, while maintaining cost efficiency.