Experience as a Compass: Multi-agent RAG with Evolving Orchestration and Agent Prompts

TL;DR

HERA is a hierarchical framework that dynamically evolves multi-agent orchestration and prompts, significantly improving performance on complex, multi-hop knowledge tasks by adaptive coordination and behavior refinement.

Contribution

It introduces a novel adaptive multi-agent RAG system with joint evolution of orchestration and prompts, surpassing static approaches in diverse reasoning tasks.

Findings

Achieves 38.69% average improvement over recent baselines.

Demonstrates emergent self-organization in multi-agent networks.

Maintains robustness and token efficiency across benchmarks.

Abstract

Multi-agent Retrieval-Augmented Generation (RAG), wherein each agent takes on a specific role, supports hard queries that require multiple steps and sources, or complex reasoning. Existing approaches, however, rely on static agent behaviors and fixed orchestration strategies, leading to brittle performance on diverse, multi-hop tasks. We identify two key limitations: the lack of continuously adaptive orchestration mechanisms and the absence of behavior-level learning for individual agents. To this end, we propose HERA, a hierarchical framework that jointly evolves multi-agent orchestration and role-specific agent prompts. At the global level, HERA optimizes query-specific agent topologies through reward-guided sampling and experience accumulation. At the local level, Role-Aware Prompt Evolution refines agent behaviors via credit assignment and dual-axes adaptation along operational and behavioral principles, enabling targeted, role-conditioned improvements. On six knowledge-intensive benchmarks, HERA achieves an average improvement of 38.69\% over recent baselines while maintaining robust generalization and token efficiency. Topological analyses reveal emergent self-organization, where sparse exploration yields compact, high-utility multi-agent networks, demonstrating both efficient coordination and robust reasoning.