Adaptive In-conversation Team Building for Language Model Agents

TL;DR

This paper introduces Captain Agent, an adaptive multi-agent framework that dynamically forms teams of LLM agents for complex tasks, significantly improving accuracy and efficiency without task-specific tuning.

Contribution

The paper presents a novel adaptive team-building paradigm with the Captain Agent design, enabling flexible, dynamic team formation and management for multi-step problem solving.

Findings

Captain Agent outperforms existing methods with 21.94% accuracy improvement.

It enhances conversation quality of weaker LLMs.

Achieves competitive performance at low cost.

Abstract

Leveraging multiple large language model (LLM) agents has shown to be a promising approach for tackling complex tasks, while the effective design of multiple agents for a particular application remains an art. It is thus intriguing to answer a critical question: Given a task, how can we build a team of LLM agents to solve it effectively? Our new adaptive team-building paradigm offers a flexible solution, realized through a novel agent design named Captain Agent. It dynamically forms and manages teams for each step of a task-solving process, utilizing nested group conversations and reflection to ensure diverse expertise and prevent stereotypical outputs, allowing for a flexible yet structured approach to problem-solving. A comprehensive evaluation across six real-world scenarios demonstrates that Captain Agent significantly outperforms existing multi-agent methods with 21.94% improvement in average accuracy, providing outstanding performance without requiring task-specific prompt engineering. Our exploration of different backbone LLM and cost analysis further shows that Captain Agent can improve the conversation quality of weak LLM and achieve competitive performance with extremely low cost, which illuminates the application of multi-agent systems.