Multi-Agent Collaboration via Evolving Orchestration

TL;DR

This paper introduces a dynamic, reinforcement learning-based orchestrator for multi-agent LLM collaboration, enabling flexible, efficient collective reasoning that adapts to task complexity and reduces computational costs.

Contribution

It proposes a novel puppeteer-style paradigm with an adaptive orchestrator trained via reinforcement learning for improved multi-agent LLM collaboration.

Findings

Achieves superior performance over static structures

Reduces computational costs in multi-agent tasks

Emergence of cyclic reasoning structures enhances reasoning quality

Abstract

Large language models (LLMs) have achieved remarkable results across diverse downstream tasks, but their monolithic nature restricts scalability and efficiency in complex problem-solving. While recent research explores multi-agent collaboration among LLMs, most approaches rely on static organizational structures that struggle to adapt as task complexity and agent numbers grow, resulting in coordination overhead and inefficiencies. To this end, we propose a puppeteer-style paradigm for LLM-based multi-agent collaboration, where a centralized orchestrator ("puppeteer") dynamically directs agents ("puppets") in response to evolving task states. This orchestrator is trained via reinforcement learning to adaptively sequence and prioritize agents, enabling flexible and evolvable collective reasoning. Experiments on closed- and open-domain scenarios show that this method achieves superior performance with reduced computational costs. Analyses further reveal that the key improvements consistently stem from the emergence of more compact, cyclic reasoning structures under the orchestrator's evolution. Our code is available at https://github.com/OpenBMB/ChatDev/tree/puppeteer.