Arbor: A Framework for Reliable Navigation of Critical Conversation Flows

TL;DR

Arbor is a modular framework that enhances the reliability and efficiency of LLM-driven decision workflows in critical domains by decomposing decision trees into node-level tasks and dynamically orchestrating model calls.

Contribution

It introduces a novel, architecture-based approach for structured decision navigation that improves accuracy, reduces latency, and lowers costs compared to monolithic prompt methods.

Findings

29.4 percentage point increase in turn accuracy

57.1% reduction in per-turn latency

14.4x decrease in per-turn cost

Abstract

Large language models struggle to maintain strict adherence to structured workflows in high-stakes domains such as healthcare triage. Monolithic approaches that encode entire decision structures within a single prompt are prone to instruction-following degradation as prompt length increases, including lost-in-the-middle effects and context window overflow. To address this gap, we present Arbor, a framework that decomposes decision tree navigation into specialized, node-level tasks. Decision trees are standardized into an edge-list representation and stored for dynamic retrieval. At runtime, a directed acyclic graph (DAG)-based orchestration mechanism iteratively retrieves only the outgoing edges of the current node, evaluates valid transitions via a dedicated LLM call, and delegates response generation to a separate inference step. The framework is agnostic to the underlying decision logic and model provider. Evaluated against single-prompt baselines across 10 foundation models using annotated turns from real clinical triage conversations. Arbor improves mean turn accuracy by 29.4 percentage points, reduces per-turn latency by 57.1%, and achieves an average 14.4x reduction in per-turn cost. These results indicate that architectural decomposition reduces dependence on intrinsic model capability, enabling smaller models to match or exceed larger models operating under single-prompt baselines.