SSR: Socratic Self-Refine for Large Language Model Reasoning

TL;DR

SSR introduces a fine-grained, step-level self-refinement framework for large language models, significantly improving reasoning accuracy and interpretability by decomposing responses into verifiable sub-steps and iteratively refining unreliable parts.

Contribution

The paper presents Socratic Self-Refine (SSR), a novel approach for detailed evaluation and correction of LLM reasoning, outperforming existing self-refinement methods on multiple benchmarks.

Findings

SSR outperforms state-of-the-art self-refinement baselines.

It provides interpretable reasoning chains.

The method enhances accuracy across five reasoning benchmarks.

Abstract

Large Language Models (LLMs) have demonstrated remarkable reasoning abilities, yet existing test-time frameworks often rely on coarse self-verification and self-correction, limiting their effectiveness on complex tasks. In this paper, we propose Socratic Self-Refine (SSR), a novel framework for fine-grained evaluation and precise refinement of LLM reasoning. Our proposed SSR decomposes model responses into verifiable (sub-question, sub-answer) pairs, enabling step-level confidence estimation through controlled re-solving and self-consistency checks. By pinpointing unreliable steps and iteratively refining them, SSR produces more accurate and interpretable reasoning chains. Empirical results across five reasoning benchmarks and three LLMs show that SSR consistently outperforms state-of-the-art iterative self-refinement baselines. Beyond performance gains, SSR provides a principled black-box approach for evaluating and understanding the internal reasoning processes of LLMs. Code is available at https://github.com/SalesforceAIResearch/socratic-self-refine-reasoning.