ORCH: many analyses, one merge—a deterministic multi-agent orchestrator for discrete-choice reasoning with EMA-guided routing
Hanlin Zhou, Huah Yong Chan
TL;DR
ORCH is a deterministic system that improves discrete-choice reasoning by using a stable routing method with multiple AI agents.
Contribution
ORCH introduces a deterministic multi-agent orchestrator using EMA-guided routing for discrete-choice reasoning.
Findings
ORCH improves accuracy over low-cost single models on discrete-choice tasks.
Deterministic routing reduces reliance on expensive models while maintaining performance.
The framework offers consistent and reproducible results across multiple runs.
Abstract
Multi-agent/ensemble approaches can improve discrete-choice reasoning with large language models, but common orchestration methods are often non-deterministic, expensive, and difficult to reproduce. We propose ORCH, a deterministic multi-agent orchestrator that targets higher accuracy and better cost–performance via stable routing. ORCH uses a pool of heterogeneous LLM agents and a deterministic routing mechanism based on exponential moving average (EMA) performance tracking. For each question, ORCH selects a small subset of agents, obtains candidate answers, and merges them through a controlled aggregation procedure. We evaluate ORCH on multiple discrete-choice benchmarks and compare against single-model baselines and non-routed ensemble strategies under consistent prompting and scoring. ORCH delivers consistent accuracy improvements over the best low-cost single model and provides…
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
Figure 18
Figure 19
Figure 20
Figure 21
Figure 22
Figure 23
Figure 24
Figure 25Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Reinforcement Learning in Robotics · Explainable Artificial Intelligence (XAI)