$S^3$: Stratified Scaling Search for Test-Time in Diffusion Language Models

TL;DR

This paper introduces $S^3$, a verifier-guided search method that reallocates compute during diffusion model denoising to improve output quality without retraining.

Contribution

The paper proposes $S^3$, a novel stratified search technique that enhances diffusion language model outputs by dynamically resampling trajectories guided by a lightweight verifier.

Findings

$S^3$ improves performance on mathematical reasoning benchmarks.

It achieves significant gains without changing the underlying model or decoding schedule.

The method effectively approximates a reward-tilted sampling distribution.

Abstract

Test-time scaling investigates whether a fixed diffusion language model (DLM) can generate better outputs when given more inference compute, without additional training. However, naive best-of-$K$ sampling is fundamentally limited because it repeatedly draws from the same base diffusion distribution, whose high-probability regions are often misaligned with high-quality outputs. We propose $S^3$ (Stratified Scaling Search), a classical verifier-guided search method that improves generation by reallocating compute during the denoising process rather than only at the final output stage. At each denoising step, $S^3$ expands multiple candidate trajectories, evaluates them with a lightweight reference-free verifier, and selectively resamples promising candidates while preserving diversity within the search frontier. This procedure effectively approximates a reward-tilted sampling distribution that favors higher-quality outputs while remaining anchored to the model prior. Experiments with LLaDA-8B-Instruct on MATH-500, GSM8K, ARC-Challenge, and TruthfulQA demonstrate that $S^3$ consistently improves performance across benchmarks, achieving the largest gains on mathematical reasoning tasks while leaving the underlying model and decoding schedule unchanged. These results show that classical search over denoising trajectories provides a practical mechanism for test-time scaling in DLMs.