More Agents Is All You Need

TL;DR

This paper introduces Agent Forest, a simple sampling-and-voting method where increasing the number of agents improves large language model performance, complementing existing methods and depending on task difficulty.

Contribution

The paper demonstrates that scaling LLMs with more agents via sampling-and-voting enhances performance, providing a straightforward approach that is orthogonal to other methods.

Findings

Performance improves with more agents across benchmarks.

The method's effectiveness correlates with task difficulty.

Code is publicly available for replication.

Abstract

We find that, simply via a sampling-and-voting method, the performance of large language models (LLMs) scales with the number of agents instantiated. Also, this method, termed as Agent Forest, is orthogonal to existing complicated methods to further enhance LLMs, while the degree of enhancement is correlated to the task difficulty. We conduct comprehensive experiments on a wide range of LLM benchmarks to verify the presence of our finding, and to study the properties that can facilitate its occurrence. Our code is publicly available at: https://github.com/MoreAgentsIsAllYouNeed/AgentForest