QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signals

TL;DR

QuantLRM introduces a novel weight quantization method for large reasoning models that leverages fine-tuning signals, improving compression efficiency and performance across multiple benchmarks.

Contribution

This paper proposes QuantLRM, a new quantization approach using weight update signals during fine-tuning, with a channel importance measure that outperforms existing methods.

Findings

QuantLRM improves quantization performance by an average of 6.55% on RL fine-tuned models.

The method is effective across various fine-tuning types and reasoning benchmarks.

Pseudo-fine-tuning signals enable QuantLRM to work well even without actual fine-tuning.

Abstract

Weight-only quantization is important for compressing Large Language Models (LLMs). Inspired by the spirit of classical magnitude pruning, we study whether the magnitude of weight updates during reasoning-incentivized fine-tuning can provide valuable signals for quantizing Large Reasoning Models (LRMs). We hypothesize that the smallest and largest weight updates during fine-tuning are more important than those of intermediate magnitude, a phenomenon we term "protecting both ends". Upon hypothesis validation, we introduce QuantLRM, which stands for weight quantization of LRMs via fine-tuning signals. We fit simple restricted quadratic functions on weight updates to protect both ends. By multiplying the average quadratic values with the count of zero weight updates of channels, we compute channel importance that is more effective than using activation or second-order information. We run QuantLRM to quantize various fine-tuned models (including supervised, direct preference optimization, and reinforcement learning fine-tuning) over four reasoning benchmarks (AIME-120, FOLIO, temporal sequences, and GPQA-Diamond) and empirically find that QuantLRM delivers a consistent improvement for LRMs quantization, with an average improvement of 6.55% on a reinforcement learning fine-tuned model. Also supporting non-fine-tuned LRMs, QuantLRM gathers effective signals via pseudo-fine-tuning, which greatly enhances its applicability.