Gradually Compacting Large Language Models for Reasoning Like a Boiling Frog

TL;DR

This paper introduces a gradual compression method for large language models that reduces size significantly while preserving reasoning performance, using iterative pruning and finetuning to avoid abrupt drops in capability.

Contribution

The paper proposes a novel iterative pruning and finetuning approach called Prune-Tune Loop (PTL) that enables effective model compression without performance loss.

Findings

Compresses models to nearly half size with minimal performance drop

Flexible application across different pruning and training strategies

Effective on reasoning, code generation, and other tasks

Abstract

Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, but their substantial size often demands significant computational resources. To reduce resource consumption and accelerate inference, it is essential to eliminate redundant parameters without compromising performance. However, conventional pruning methods that directly remove such parameters often lead to a dramatic drop in model performance in reasoning tasks, and require extensive post-training to recover the lost capabilities. In this work, we propose a gradual compacting method that divides the compression process into multiple fine-grained iterations, applying a Prune-Tune Loop (PTL) at each stage to incrementally reduce model size while restoring performance with finetuning. This iterative approach-reminiscent of the "boiling frog" effect-enables the model to be progressively compressed without abrupt performance loss. Experimental results show that PTL can compress LLMs to nearly half their original size with only lightweight post-training, while maintaining performance comparable to the original model on reasoning tasks. Moreover, PTL is flexible and can be applied to various pruning strategies, such as neuron pruning and layer pruning, as well as different post-training methods, including continual pre-training and reinforcement learning. Additionally, experimental results confirm the effectiveness of PTL on a variety of tasks beyond mathematical reasoning, such as code generation, demonstrating its broad applicability.