Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic

TL;DR

This paper introduces Timely Machine, a framework for dynamic, time-aware test-time scaling of language models in agentic scenarios, emphasizing wall-clock time and adaptive strategies.

Contribution

It redefines test-time as wall-clock time, introduces a new benchmark, and proposes reinforcement learning for models to adapt reasoning based on time budgets.

Findings

Smaller models perform better with fast feedback and frequent interactions.

Larger models excel in high-latency settings due to better interaction quality.

Timely-RL improves models' awareness of time constraints and enhances performance.

Abstract

As large language models (LLMs) increasingly tackle complex reasoning tasks, test-time scaling has become critical for enhancing capabilities. However, in agentic scenarios with frequent tool calls, the traditional generation-length-based definition breaks down: tool latency decouples inference time from generation length. We propose Timely Machine, redefining test-time as wall-clock time, where models dynamically adjust strategies based on time budgets. We introduce Timely-Eval, a benchmark spanning high-frequency tool calls, low-frequency tool calls, and time-constrained reasoning. By varying tool latency, we find smaller models excel with fast feedback through more interactions, while larger models dominate high-latency settings via superior interaction quality. Moreover, existing models fail to adapt reasoning to time budgets. We propose Timely-RL to address this gap. After cold-start supervised fine-tuning, we use reinforcement learning to enhance temporal planning. Timely-RL improves time budget awareness and consistently boosts performance across Timely-Eval. We hope our work offers a new perspective on test-time scaling for the agentic era.