Learning to Configure Agentic AI Systems

TL;DR

This paper introduces ARC, a hierarchical policy that dynamically configures agent systems for each query, significantly improving reasoning and tool-use accuracy over fixed configurations.

Contribution

The paper formulates agent configuration as a semi-Markov decision process and presents ARC, a novel method for per-query agent configuration in LLM-based systems.

Findings

ARC increases reasoning accuracy by 31.3%.

ARC improves tool-use accuracy by 13.95%.

ARC doubles success rate on the { au}-Bench Airline benchmark.

Abstract

Configuring LLM-based agent systems involves choosing workflows, tools, token budgets, and prompts from a large combinatorial design space, and is typically handled today by fixed templates or hand-tuned heuristics that apply the same configuration regardless of query difficulty, leading to brittle behavior and wasted compute. To address this, we formulate agent configuration as a semi-Markov decision process (SMDP) where each configuration acts as a temporally extended option that determines how an agent system processes a query, and introduce introduce ARC (Agentic Resource & Configuration learner), a lightweight hierarchical policy that dynamically selects query-specific agent configurations. Across reasoning, tool-use, and agentic benchmarks, ARC consistently improves over budget-matched tool-augmented LLMs, increasing average reasoning accuracy by 31.3%, tool-use accuracy by 13.95%, and doubling {\tau}-Bench (Airline) Pass^1 success from 9.0% to 18.0%. These results demonstrate that learning per-query agent configurations is a powerful alternative to "one size fits all" designs.