HiveMind: Contribution-Guided Online Prompt Optimization of LLM Multi-Agent Systems

TL;DR

HiveMind introduces a contribution-guided online prompt optimization framework for LLM multi-agent systems, utilizing a novel DAG-Shapley algorithm to efficiently quantify agent contributions and improve collaborative performance.

Contribution

The paper presents DAG-Shapley, an efficient attribution algorithm leveraging DAG structures to reduce computational costs of Shapley values in multi-agent systems.

Findings

DAG-Shapley reduces LLM calls by over 80%.

HiveMind outperforms static baselines in stock-trading scenarios.

DAG-Shapley maintains attribution accuracy comparable to full Shapley values.

Abstract

Recent advances in LLM-based multi-agent systems have demonstrated remarkable capabilities in complex decision-making scenarios such as financial trading and software engineering. However, evaluating each individual agent's effectiveness and online optimization of underperforming agents remain open challenges. To address these issues, we present HiveMind, a self-adaptive framework designed to optimize LLM multi-agent collaboration through contribution analysis. At its core, HiveMind introduces Contribution-Guided Online Prompt Optimization (CG-OPO), which autonomously refines agent prompts based on their quantified contributions. We first propose the Shapley value as a grounded metric to quantify each agent's contribution, thereby identifying underperforming agents in a principled manner for automated prompt refinement. To overcome the computational complexity of the classical Shapley value, we present DAG-Shapley, a novel and efficient attribution algorithm that leverages the inherent Directed Acyclic Graph structure of the agent workflow to axiomatically prune non-viable coalitions. By hierarchically reusing intermediate outputs of agents in the DAG, our method further reduces redundant computations, and achieving substantial cost savings without compromising the theoretical guarantees of Shapley values. Evaluated in a multi-agent stock-trading scenario, HiveMind achieves superior performance compared to static baselines. Notably, DAG-Shapley reduces LLM calls by over 80\% while maintaining attribution accuracy comparable to full Shapley values, establishing a new standard for efficient credit assignment and enabling scalable, real-world optimization of multi-agent collaboration.