Language Model Teams as Distributed Systems

TL;DR

This paper proposes a distributed systems framework to analyze and optimize large language model teams, addressing key questions about their effectiveness, structure, and comparison to single agents.

Contribution

It introduces a principled approach using distributed systems theory to evaluate LLM teams, bridging insights from distributed computing and language models.

Findings

Distributed systems principles apply to LLM teams.

Team structure impacts performance and utility.

Fundamental challenges mirror those in distributed computing.

Abstract

Large language models (LLMs) are growing increasingly capable, prompting recent interest in LLM teams. Yet, despite increased deployment of LLM teams at scale, we lack a principled framework for addressing key questions such as when a team is helpful, how many agents to use, how structure impacts performance -- and whether a team is better than a single agent. Rather than designing and testing these possibilities through trial-and-error, we propose using distributed systems as a principled foundation for creating and evaluating LLM teams. We find that many of the fundamental advantages and challenges studied in distributed computing also arise in LLM teams, highlighting the rich practical insights that can come from the cross-talk of these two fields of study.