Scaling Large Language Model-based Multi-Agent Collaboration

TL;DR

This paper introduces MacNet, a multi-agent collaboration network that scales with many agents, demonstrating a collaborative scaling law where performance improves logistically as agents increase, enhancing collective reasoning.

Contribution

It proposes a novel multi-agent collaboration architecture using directed acyclic graphs and uncovers a collaborative scaling law in large language model-based multi-agent systems.

Findings

Supports over a thousand agents in collaboration.

Irregular topologies outperform regular ones.

Performance follows a logistic growth pattern with scaling.

Abstract

Recent breakthroughs in large language model-driven autonomous agents have revealed that multi-agent collaboration often surpasses each individual through collective reasoning. Inspired by the neural scaling law--increasing neurons enhances performance, this study explores whether the continuous addition of collaborative agents can yield similar benefits. Technically, we utilize directed acyclic graphs to organize agents into a multi-agent collaboration network (MacNet), upon which their interactive reasoning is topologically orchestrated for autonomous task solving. Extensive evaluations reveal that it effectively supports collaboration among over a thousand agents, with irregular topologies outperforming regular ones. We also identify a collaborative scaling law--the overall performance follows a logistic growth pattern as agents scale, with collaborative emergence occurring earlier than traditional neural emergence. We speculate this may be because scaling agents catalyzes their multidimensional considerations during interactive reflection and refinement, thereby producing more comprehensive artifacts. The code is available at https://github.com/OpenBMB/ChatDev/tree/macnet.