PACE: Prefix-Protected and Difficulty-Aware Compression for Efficient Reasoning

TL;DR

PACE introduces a hierarchical, difficulty-aware compression method for language reasoning models that reduces token usage significantly while improving accuracy across various domains.

Contribution

It proposes a novel dual-level framework that dynamically balances reasoning trace conciseness and exploration based on query difficulty, addressing overthinking in LRMs.

Findings

Reduces token usage by up to 55.7%.

Improves accuracy by up to 4.1%.

Effective across math, code, science, and general domains.

Abstract

Language Reasoning Models (LRMs) achieve strong performance by scaling test-time computation but often suffer from ``overthinking'', producing excessively long reasoning traces that increase latency and memory usage. Existing LRMs typically enforce conciseness with uniform length penalties, which over-compress crucial early deduction steps at the sequence level and indiscriminately penalize all queries at the group level. To solve these limitations, we propose \textbf{\model}, a dual-level framework for prefix-protected and difficulty-aware compression under hierarchical supervision. At the sequence level, prefix-protected optimization employs decaying mixed rollouts to maintain valid reasoning paths while promoting conciseness. At the group level, difficulty-aware penalty dynamically scales length constraints based on query complexity, maintaining exploration for harder questions while curbing redundancy on easier ones. Extensive experiments on DeepSeek-R1-Distill-Qwen (1.5B/7B) demonstrate that \model achieves a substantial reduction in token usage (up to \textbf{55.7\%}) while simultaneously improving accuracy (up to \textbf{4.1\%}) on math benchmarks, with generalization ability to code, science, and general domains.