TRiMS: Real-Time Tracking of Minimal Sufficient Length for Efficient Reasoning via RL

TL;DR

TRiMS introduces a theoretical metric and a novel training method to optimize reasoning chain length in large language models, significantly reducing token usage while maintaining or improving accuracy.

Contribution

It proposes MSL, a rigorous metric for minimal reasoning length, and develops TRiMS, a training approach that leverages MSL for efficient reasoning in language models.

Findings

Achieves over 80% reduction in reasoning tokens.

Maintains or improves accuracy across benchmarks.

Provides the first measurable lower bound for reasoning-chain compression.

Abstract

Large language models achieve breakthroughs in complex reasoning via long chain-of-thought sequences. However, this often leads to severe reasoning inflation, causing substantial computational redundancy. To maximize Intelligence per Token, we introduce a theoretical metric, MSL-Minimal Sufficient Length. MSL rigorously characterizes the shortest reasoning length that preserves answer correctness. We provide a recursive definition based on independently sampled sequences and prove the existence of its limit, establishing the first measurable lower bound for reasoning-chain compression. Building on an analysis of mainstream CoT compression strategies, we identify key structural factors enabling a model to approach MSL. Based on these insights, we propose TRiMS which employs the GRPO algorithm in conjunction with MSL-based estimation during training, while mitigating instabilities during the training process through dynamic batch aggregation and advantage computation using batch-level standard deviation. TRiMS achieves over 80% CoT token reduction with a minor accuracy boost across all benchmarks.