Dyve: Thinking Fast and Slow for Dynamic Process Verification

TL;DR

Dyve is a dynamic process verifier that combines fast and slow reasoning methods, using a novel supervision technique to improve large language model verification accuracy on complex tasks.

Contribution

It introduces a novel adaptive reasoning framework inspired by Kahneman's theory, with a step-wise consensus filtering method for high-quality supervision signals.

Findings

Outperforms existing process-based verifiers on ProcessBench and MATH datasets.

Significantly improves performance in Best-of-N settings.

Demonstrates effective integration of fast and slow reasoning in LLM verification.

Abstract

We present Dyve, a dynamic process verifier that enhances reasoning error detection in large language models by integrating fast and slow thinking, inspired by Kahneman's Systems Theory. Dyve adaptively applies immediate token-level confirmation System 1 for straightforward steps and comprehensive analysis System 2 for complex ones. Leveraging a novel step-wise consensus-filtered process supervision technique, combining Monte Carlo estimation with LLM based evaluation, Dyve curates high-quality supervision signals from noisy data. Experimental results on ProcessBench and the MATH dataset confirm that Dyve significantly outperforms existing process-based verifiers and boosts performance in Best-of-N settings.