CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning

TL;DR

CRISP is a framework that compresses chain-of-thought reasoning by leveraging intrinsic saliency signals, significantly reducing token count while maintaining accuracy.

Contribution

It introduces a novel saliency-based pruning method that aligns compression with the model's internal reasoning dynamics, outperforming prior external compression approaches.

Findings

Achieves 50-60% token reduction without accuracy loss

Identifies reasoning termination token as an information anchor

Demonstrates effectiveness across various models and datasets

Abstract

Long Chain-of-Thought (CoT) reasoning is pivotal for the success of recent reasoning models but suffers from high computational overhead and latency. While prior works attempt to compress CoT via external compressor, they often fail to align with the model's internal reasoning dynamics, resulting in the loss of critical logical steps. This paper presents \textbf{C}ompressing \textbf{R}edundancy in Chain-of-Thought via \textbf{I}ntrinsic \textbf{S}aliency \textbf{P}runing (\textbf{CRISP}), a framework that compresses CoT by exploiting the model's intrinsic saliency. Our analysis reveals a distinct phenomenon: the reasoning termination token \texttt{[object Object]} acts as an information anchor, where its attention pattern effectively demarcates essential reasoning from redundancy. Based on this finding, we design a policy that utilizes these intrinsic attention signals to guide atomic compression operations. In contrast to coarse-grained pruning strategies, CRISP strategically distills the reasoning chain to maximize information density while preserving logical coherence. Empirical results across various backbone models and mathematical datasets demonstrate that CRISP achieves a 50-60% reduction in token count without compromising accuracy, effectively mitigating the efficiency bottleneck of long-context reasoning. We open-source our implementation to facilitate further research in efficient reasoning.