Stepwise Think-Critique: A Unified Framework for Robust and Interpretable LLM Reasoning

TL;DR

This paper introduces Stepwise Think-Critique (STC), a unified framework enabling large language models to perform integrated reasoning and self-critique at each step, improving interpretability and robustness in complex problem solving.

Contribution

The paper presents a novel end-to-end trainable model that interleaves reasoning and critique, using hybrid reinforcement learning to enhance correctness and self-evaluation in LLMs.

Findings

STC outperforms existing models on mathematical reasoning benchmarks.

It produces more interpretable and reliable reasoning traces.

Demonstrates strong critical-thinking capabilities in LLMs.

Abstract

Human beings solve complex problems through critical thinking, where reasoning and evaluation are intertwined to converge toward correct solutions. However, most existing large language models (LLMs) treat the reasoning and verification as separate processes: they either generate reasoning without explicit self-checking or rely on external verifiers to detect errors post hoc. The former lacks immediate feedback, while the latter increases system complexity and hinders synchronized learning. Motivated by human critical thinking, we propose Stepwise Think-Critique (STC), a unified and end-to-end trainable framework that interleaves reasoning and self-critique at every intermediate step within a single model. STC is trained with a hybrid reinforcement learning objective that integrates reasoning rewards and critique-consistency rewards, thereby jointly optimizing solution correctness and reliability of self-evaluation. Experiments on mathematical reasoning benchmarks show that STC demonstrates strong critical-thinking capabilities and produces more interpretable reasoning traces, representing a step toward LLMs with built-in critical thinking.