A Scalable Communication Protocol for Networks of Large Language Models
Samuele Marro, Emanuele La Malfa, Jesse Wright, Guohao Li, Nigel, Shadbolt, Michael Wooldridge, Philip Torr
TL;DR
This paper introduces Agora, a scalable, decentralized communication protocol for large language model networks that efficiently balances standard routines, natural language, and LLM-generated routines, enabling autonomous problem-solving.
Contribution
The paper presents Agora, a novel meta protocol that overcomes the Agent Communication Trilemma, facilitating scalable, robust, and autonomous communication among large language model agents.
Findings
Emergence of self-organising, fully automated protocols
Achieves complex goals without human intervention
Handles interface and member changes robustly
Abstract
Communication is a prerequisite for collaboration. When scaling networks of AI-powered agents, communication must be versatile, efficient, and portable. These requisites, which we refer to as the Agent Communication Trilemma, are hard to achieve in large networks of agents. We introduce Agora, a meta protocol that leverages existing communication standards to make LLM-powered agents solve complex problems efficiently. In Agora, agents typically use standardised routines for frequent communications, natural language for rare communications, and LLM-written routines for everything in between. Agora sidesteps the Agent Communication Trilemma and robustly handles changes in interfaces and members, allowing unprecedented scalability with full decentralisation and minimal involvement of human beings. On large Agora networks, we observe the emergence of self-organising, fully automated…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
1.Designing efficient communication protocol is an important research problem to scale up multi-agent systems powered by LLMs. 2.The motivation is well-articulated, aligning with current trends in agentic AI and the need for scalable solutions. 3.Research of communication protocol could have broad implications for multi-agent systems.
1.The empirical experiments are insufficient. Only Figure 5 has some empirical results. 2.The proposed method is not compared against any baseline methods. 3.No theoretical analysis is provided for the communication cost and performance. Based on the above observations, the present work does not contain enough technical contribution to be published as a research paper. The author may consider to submit as a position paper.
1. The identification of the LLM-powered agent communication trilemma. 2. The proposed protocal is novel, which employs a hybrid approach, using standardized routines for frequent communications, natural language for rare communications, and LLM-written routines for everything in between. 3. The paper is well organized and easy to follow. 4. The empirical validation presented is robust, showcasing Agora's scalability and cost-effectiveness through extensive testing involving a network of 100
1. A primary concern regarding the paper is its alignment with the focus of the conference. The content appears to be more closely aligned with themes typically found in communication-focused conferences, which may limit its relevance to the current audience. 2. Some details are missing: a. what's the system prompt to guide the negotiation of the protocol in the demo mentioned in section 5.2? b. The paper does not provide a thorough analysis of the errors encountered during the demos and h
Good motivation of the trilemma problem, and timely submission given the interest in agent networks and LLMs. The authors present the trilemma problem in detail (maybe almost too much detail, because other important things then don't have enough space), thus giving a framework to analyse various solutions. Agora works pragmatically by using efficient protocols for frequent information while using natural language for infrequent requests Agora does not rely on a single provider for any of the n
No theoretical consideration of the problem: What is the expected cost of various solutions? Lacking more experimental evaluations: - Under various network sizes and configurations - Under various communication distributions. - Everyone talks frequently to everyone uniformly - Everyone talks frequently to a few agents - Only a few agents communicate frequently with a few agents - Private protocol databases for each agent - What is the impact of changing the order of the queries? How large can
The domain tackled in the paper is for sure of interest: indeed the emergence of communication modalities and protocols between a group of LLM-based agents is a very valuable topic in the domain of cooperative AI and cooperative LLMs.
- The main contribution of the paper is the implementation of a communication protocol between heterogenous LLM-based agents. So, for sure an interesting engineering contribution but not a good fit for a machine learning conference; - The hypothesis is that a network of heterogeneous LLMs can automate a variety of complex tasks. However, this hypothesis does not seem neither so central and neither evaluated seriously and quantitatively in the current version of the paper.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCognitive Computing and Networks · Distributed and Parallel Computing Systems · Semantic Web and Ontologies