Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoning

TL;DR

This paper explores whether pruning reasoning steps in long chain-of-thought prompts can enhance reasoning accuracy and efficiency in language models by aligning reasoning complexity with model capacity.

Contribution

It introduces Prune-on-Logic, a structure-aware pruning framework that selectively removes low-utility reasoning steps to improve reasoning performance in small language models.

Findings

Verification pruning improves accuracy and reduces token usage.

Pruning reasoning steps degrades performance, indicating the importance of selective pruning.

Larger models benefit more from pruning due to richer reasoning capabilities.

Abstract

Long chain-of-thought (Long-CoT) reasoning improves accuracy in LLMs, yet its verbose, self-reflective style often hinders effective distillation into small language models (SLMs). We revisit Long-CoT compression through the lens of capability alignment and ask: Can pruning improve reasoning? We propose Prune-on-Logic, a structure-aware framework that transforms Long-CoT into logic graphs and selectively prunes low-utility reasoning steps under self-verification constraints. Through systematic analysis across three pruning strategies targeting entire chains, core reasoning, and verification, we find that verification pruning consistently improves accuracy while reducing token usage, whereas pruning reasoning steps or indiscriminate pruning degrades performance. Our study reveals that effective pruning aligns supervision with model capacity rather than merely shortening inputs. Gains hold across tasks, model scales, and CoT capability, with larger models benefiting more from pruning due to richer but more redundant reasoning. Our empirical findings highlight pruning as a structural optimization strategy for aligning CoT reasoning with SLM capacity.