interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification

TL;DR

interwhen is a versatile test-time verification framework that guides reasoning models using intermediate feedback and automatically generated verifiers, significantly enhancing accuracy and policy adherence.

Contribution

It introduces a novel, plug-and-play verification system that monitors reasoning traces and synthesizes verifiers from natural language policies, improving model reliability without fine-tuning.

Findings

Achieves near-perfect accuracy on reasoning benchmarks with minimal token usage.

Improves task completion rates from 32% to 87% on telecom domain benchmarks.

Enables policy compliance and correctness in reasoning models through automatic verifier synthesis.

Abstract

Reasoning models produce long traces of intermediate decisions and tool calls, making test-time verification important for ensuring correctness. Existing approaches either verify only the final answer, which misses early errors, or rely on branch-and-verify strategies that explore multiple trajectories. We introduce interwhen, a single-trajectory verification framework that steers model behavior by providing feedback on intermediate reasoning traces. It addresses two key challenges. First, given a set of verifiers, obtaining verifiable states from the reasoning trace typically requires prompt engineering or external task decomposition into fixed steps. Instead, we propose a monitoring system that periodically polls the reasoning trace and forks inference of the reasoning model to recover intermediate states. Verifiers are run asynchronously alongside generation, adding negligible overhead on correct executions and intervening only when violations occur. Second, beyond math and code, a central challenge for process verification is the scarcity of verifiers. interwhen addresses this through automatic verifier synthesis from natural-language policy documents. Given a policy, it can generate code-based verifiers, including provably correct verifiers in Lean and z3. Together, these contributions yield a plug-and-play test-time verification system that can improve task completion and policy compliance of any reasoning agent. On reasoning benchmarks where policies encode mathematical or logical constraints, interwhen achieves near-perfect accuracy for reasoning models using a fraction of the tokens of baselines. On agentic benchmarks with policy-based verifier generation, it enables improvements in task quality for SLMs without any finetuning, e.g., task completion rate of Qwen3-30B jumps from 32% to 87% on the telecom domain in tau2-bench. Code at https://github.com/microsoft/interwhen.