Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scaling

TL;DR

This paper introduces DORA, a novel resource allocation method for test-time scaling of large language models, optimizing compute use during search to improve accuracy on mathematical reasoning benchmarks.

Contribution

The paper formulates test-time search as a resource allocation problem and proposes DORA, the first provably optimal method that decouples direction quality from candidate count for efficient compute use.

Findings

DORA outperforms baselines on mathematical reasoning benchmarks.

DORA achieves state-of-the-art accuracy with comparable computational cost.

Optimal resource allocation improves test-time reasoning efficiency.

Abstract

Test-Time Scaling (TTS) improves the performance of Large Language Models (LLMs) by using additional inference-time computation to explore multiple reasoning paths through search. Yet how to allocate a fixed rollout budget most effectively during search remains underexplored, often resulting in inefficient use of compute at test time. To bridge this gap, we formulate test-time search as a resource allocation problem and derive the optimal allocation strategy that maximizes the probability of obtaining a correct solution under a fixed rollout budget. Within this formulation, we reveal a core limitation of existing search methods: solution-level allocation tends to favor reasoning directions with more candidates, leading to theoretically suboptimal and inefficient use of compute. To address this, we propose Direction-Oriented Resource Allocation (DORA), a provably optimal method that mitigates this bias by decoupling direction quality from candidate count and allocating resources at the direction level. To demonstrate DORA's effectiveness, we conduct extensive experiments on challenging mathematical reasoning benchmarks including MATH500, AIME2024, and AIME2025. The empirical results show that DORA consistently outperforms strong baselines with comparable computational cost, achieving state-of-the-art accuracy. We hope our findings contribute to a broader understanding of optimal TTS for LLMs.