Parallel Test-Time Scaling with Multi-Sequence Verifiers

TL;DR

This paper introduces the Multi-Sequence Verifier (MSV), a novel joint-processing verifier for parallel test-time scaling in large language models, improving answer selection and reducing latency through early-stopping strategies.

Contribution

The paper proposes MSV, the first verifier to jointly process multiple candidates, enhancing calibration and enabling efficient early-stopping in parallel decoding scenarios.

Findings

MSV improves answer selection accuracy.

MSV reduces latency by approximately 50% with maintained accuracy.

Joint processing of candidates outperforms isolated scoring methods.

Abstract

Parallel test-time scaling, which generates multiple candidate solutions for a single problem, is a powerful technique for improving large language model performance. However, it is hindered by two key bottlenecks: accurately selecting the correct solution from the candidate pool, and the high inference latency from generating many full solutions. We argue that both challenges are fundamentally linked to verifier calibration. A well-calibrated verifier not only improves answer selection, but also enables early-stopping strategies to reduce latency. However, existing verifiers are limited as they score each candidate in isolation, overlooking rich contextual information across the set of candidates. To address this, we introduce the Multi-Sequence Verifier (MSV), the first verifier designed to jointly process all candidate solutions and model their interactions. MSV achieves improved calibration, which directly enhances best-of-N selection performance. We further introduce a streaming MSV variant that empowers a novel early-stopping framework. Our novel framework fully leverages parallel decoding, which contrasts with the existing multi-sequence early exit works that decode sequences one by one and thus incur significant latency. In this novel setting, MSV can achieve the same target accuracy with around half the latency that would be required with its counterpart that scores each solution in isolation.