Hive: A Multi-Agent Infrastructure for Algorithm- and Task-Level Scaling

TL;DR

Hive introduces a multi-agent infrastructure that enhances algorithm- and task-level scaling of large language models by optimizing resource allocation and reducing redundant computation.

Contribution

The paper presents Hive, a novel infrastructure with mechanisms for efficient resource management and redundancy reduction in multi-agent, scalable language model systems.

Findings

Logits Cache achieves 1.11x-1.76x speedup in re-sampling.

Agent-Aware Scheduling reduces hotspot miss rate by 33%-51%.

Hive enables scalable and efficient multi-agent language model deployment.

Abstract

Large language models are increasingly deployed as complex agentic systems that scale with task complexity. While prior work has extensively explored model- and system-level scaling, algorithm- and task-level scaling remain largely unaddressed, constraining the full potential of agentic systems. At the algorithm level, allocating additional inference-time computation can enhance workflow capacity but introduces cross-path redundancy: overlapping computations across multiple reasoning branches. At the task level, complex tasks can be decomposed into subproblems and delegated across multiple agents for improved scalability and parallelism. However, existing infrastructures' scheduling is unaware of the existence of multiple agents, missing opportunities to optimize resource allocation. We propose Hive, a multi-agent infrastructure that enables algorithm- and task-level scaling. Hive features a description frontend that captures per-agent behavior and supports test-time scaling algorithms. Leveraging this specification, our backend introduces two key mechanisms: Logits Cache that reuses intermediate logits across redundant sampling paths to mitigate cross-path redundancy at the algorithm level, and Agent-Aware Scheduling that efficiently allocates compute and KV-cache resources according to agent contributions at the task level. Experiments show that Logits Cache achieves an average speedup of $1.11\times$-$1.76\times$ for re-sampling, and Agent-Aware Scheduling reduces the hotspot miss rate by $33\%$-$51\%$.