Verified Multi-Agent Orchestration: A Plan-Execute-Verify-Replan Framework for Complex Query Resolution

TL;DR

VMAO is a framework that orchestrates multiple specialized LLM agents to collaboratively answer complex queries by decomposing, executing, verifying, and replanning, significantly improving answer completeness and quality.

Contribution

It introduces a verification-driven iterative loop for multi-agent orchestration, enabling adaptive replanning and improved answer quality in complex query resolution.

Findings

Answer completeness improved from 3.1 to 4.2

Source quality increased from 2.6 to 4.1

Effective verification enhances multi-agent coordination

Abstract

We present Verified Multi-Agent Orchestration (VMAO), a framework that coordinates specialized LLM-based agents through a verification-driven iterative loop. Given a complex query, our system decomposes it into a directed acyclic graph (DAG) of sub-questions, executes them through domain-specific agents in parallel, verifies result completeness via LLM-based evaluation, and adaptively replans to address gaps. The key contributions are: (1) dependency-aware parallel execution over a DAG of sub-questions with automatic context propagation, (2) verification-driven adaptive replanning that uses an LLM-based verifier as an orchestration-level coordination signal, and (3) configurable stop conditions that balance answer quality against resource usage. On 25 expert-curated market research queries, VMAO improves answer completeness from 3.1 to 4.2 and source quality from 2.6 to 4.1 (1-5 scale) compared to a single-agent baseline, demonstrating that orchestration-level verification is an effective mechanism for multi-agent quality assurance.