Socratic-PRMBench: Benchmarking Process Reward Models with Systematic Reasoning Patterns

TL;DR

Socratic-PRMBench is a comprehensive benchmark designed to evaluate process reward models across six reasoning patterns, revealing significant weaknesses in current models and aiming to guide future improvements in systematic reasoning evaluation.

Contribution

This paper introduces Socratic-PRMBench, the first benchmark systematically evaluating PRMs under diverse reasoning patterns, filling a critical gap in existing evaluation methods.

Findings

Current PRMs perform poorly across various reasoning patterns.

Existing benchmarks mainly assess stepwise correctness, not reasoning patterns.

Socratic-PRMBench contains 2995 flawed reasoning paths across six patterns.

Abstract

Process Reward Models (PRMs) are crucial in complex reasoning and problem-solving tasks (e.g., LLM agents with long-horizon decision-making) by verifying the correctness of each intermediate reasoning step. In real-world scenarios, LLMs may apply various reasoning patterns (e.g., decomposition) to solve a problem, potentially suffering from errors under various reasoning patterns. Therefore, PRMs are required to identify errors under various reasoning patterns during the reasoning process. However, existing benchmarks mainly focus on evaluating PRMs with stepwise correctness, ignoring a systematic evaluation of PRMs under various reasoning patterns. To mitigate this gap, we introduce Socratic-PRMBench, a new benchmark to evaluate PRMs systematically under six reasoning patterns, including Transformation, Decomposition, Regather, Deduction, Verification, and Integration. Socratic-PRMBench}comprises 2995 reasoning paths with flaws within the aforementioned six reasoning patterns. Through our experiments on both PRMs and LLMs prompted as critic models, we identify notable deficiencies in existing PRMs. These observations underscore the significant weakness of current PRMs in conducting evaluations on reasoning steps under various reasoning patterns. We hope Socratic-PRMBench can serve as a comprehensive testbed for systematic evaluation of PRMs under diverse reasoning patterns and pave the way for future development of PRMs.