Utility-Guided Agent Orchestration for Efficient LLM Tool Use

TL;DR

This paper introduces a utility-guided orchestration policy for tool-using LLM agents, balancing answer quality and execution cost, and providing a controllable framework for studying trade-offs in agent behavior.

Contribution

It proposes an explicit decision-making framework for agent orchestration that balances gain, cost, and redundancy, improving control over tool use in LLM agents.

Findings

Explicit orchestration signals influence agent behavior significantly.

Lightweight utility design effectively manages quality-cost trade-offs.

The framework is versatile across various agent workflows.

Abstract

Tool-using large language model (LLM) agents often face a fundamental tension between answer quality and execution cost. Fixed workflows are stable but inflexible, while free-form multi-step reasoning methods such as ReAct may improve task performance at the expense of excessive tool calls, longer trajectories, higher token consumption, and increased latency. In this paper, we study agent orchestration as an explicit decision problem rather than leaving it entirely to prompt-level behavior. We propose a utility-guided orchestration policy that selects among actions such as respond, retrieve, tool call, verify, and stop by balancing estimated gain, step cost, uncertainty, and redundancy. Our goal is not to claim universally best task performance, but to provide a controllable and analyzable policy framework for studying quality-cost trade-offs in tool-using LLM agents. Experiments across direct answering, threshold control, fixed workflows, ReAct, and several policy variants show that explicit orchestration signals substantially affect agent behavior. Additional analyses on cost definitions, workflow fairness, and redundancy control further demonstrate that lightweight utility design can provide a defensible and practical mechanism for agent control.