Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agents

TL;DR

This paper introduces BAVT, a training-free, inference-time framework for LLM agents that optimizes reasoning efficiency under strict budget constraints, outperforming brute-force methods by intelligent resource management.

Contribution

BAVT is a novel, parameter-free, budget-aware reasoning method that dynamically guides multi-hop reasoning within a single LLM, with theoretical guarantees and superior performance under low-resource conditions.

Findings

BAVT outperforms baseline methods on multiple QA benchmarks.

It achieves better results with only a quarter of the resource budget.

The approach provides a theoretical guarantee of convergence.

Abstract

Test-time scaling has become a dominant paradigm for improving LLM agent reliability, yet current approaches treat compute as an abundant resource, allowing agents to exhaust token and tool budgets on redundant steps or dead-end trajectories. Existing budget-aware methods either require expensive fine-tuning or rely on coarse, trajectory-level heuristics that cannot intervene mid-execution. We propose the Budget-Aware Value Tree (BAVT), a training-free inference-time framework that models multi-hop reasoning as a dynamic search tree guided by step-level value estimation within a single LLM backbone. Another key innovation is a budget-conditioned node selection mechanism that uses the remaining resource ratio as a natural scaling exponent over node values, providing a principled, parameter-free transition from broad exploration to greedy exploitation as the budget depletes. To combat the well-known overconfidence of LLM self-evaluation, BAVT employs a residual value predictor that scores relative progress rather than absolute state quality, enabling reliable pruning of uninformative or redundant tool calls. We further provide a theoretical convergence guarantee, proving that BAVT reaches a terminal answer with probability at least $1-\epsilon$ under an explicit finite budget bound. Extensive evaluations on four multi-hop QA benchmarks across two model families demonstrate that BAVT consistently outperforms parallel sampling baselines. Most notably, BAVT under strict low-budget constraints surpasses baseline performance at $4\times$ the resource allocation, establishing that intelligent budget management fundamentally outperforms brute-force compute scaling.