GR-Ben: A General Reasoning Benchmark for Evaluating Process Reward Models

TL;DR

GR-Ben is a comprehensive benchmark designed to evaluate process reward models across diverse reasoning domains, revealing current limitations in error detection capabilities of PRMs and LLMs beyond mathematical reasoning.

Contribution

Introduces GR-Ben, a new benchmark for assessing PRMs across science and logic domains, addressing the lack of diverse reasoning scenario evaluations.

Findings

PRMs and LLMs perform weaker in non-mathematical reasoning domains.

PRMs struggle more with knowledge-based errors, LLMs with computational errors.

The benchmark covers 22 models across multiple subdomains.

Abstract

Currently, process reward models (PRMs) have exhibited remarkable potential for test-time scaling. Since large language models (LLMs) regularly generate flawed intermediate reasoning steps when tackling a broad spectrum of reasoning and decision-making tasks, PRMs are required to possess capabilities for detecting process-level errors in real-world scenarios. However, existing benchmarks primarily focus on mathematical reasoning, thereby failing to comprehensively evaluate the error detection ability of PRMs across diverse reasoning scenarios. To mitigate this gap, we introduce GR-Ben, a process-level benchmark specifically designed for assessing PRM's performance across two primary reasoning domains (science and logic) and nine subdomains. We conduct extensive experiments on a diverse set of 22 models, encompassing both PRMs and LLMs, and derive two key findings: (1) In domains beyond mathematical reasoning, the error-detection ability of existing PRMs and LLMs is found to be markedly weaker by comparison.(2) In general, PRMs are less adept at identifying knowledge-based errors, whereas LLMs exhibit poorer performance in detecting computational errors. We hope GR-Ben can foster future researches on PRMs for general domains, thereby enhancing the reasoning capabilities of LLMs.