HAS-VQ: Hessian-Adaptive Sparse Vector Quantization for High-Fidelity LLM Compression

TL;DR

HAS-VQ introduces a Hessian-adaptive sparse vector quantization method that significantly improves high-fidelity compression of large language models by decoupling sensitive outliers and employing residual correction, outperforming standard quantization.

Contribution

The paper presents a novel Hessian-adaptive sparse vector quantization framework that effectively decouples sensitive outliers and employs residual correction for superior LLM compression.

Findings

Achieves 2.3x reduction in model size while maintaining perplexity.

Outperforms INT4 baseline in perplexity at similar bit-per-parameter.

Demonstrates Pareto dominance over standard integer quantization methods.

Abstract

Post-training quantization is essential for deploying Large Language Models (LLMs) on resource-constrained devices. However, standard integer quantization (e.g., INT4) fundamentally degrades performance by imposing a uniform grid on the heavy-tailed distribution of weight parameters, particularly in smaller-scale models (e.g., <2B parameters). We introduce HAS-VQ (Hessian-Adaptive Sparse Vector Quantization), a compression framework that strictly decouples high-sensitivity outliers from the bulk weight distribution using second-order sensitivity analysis. HAS-VQ employs a Hessian-Masked Decoupling strategy to isolate sensitive parameters, followed by robust Vector Quantization (VQ) of the remaining dense body. Crucially, we introduce a residual sparse feedback mechanism that corrects quantization errors in the most sensitive dimensions, ensuring exact reconstruction of outliers. We evaluate HAS-VQ on SmolLM2-1.7B, demonstrating two distinct regimes of superiority: (1) Pareto Dominance over Integer Baselines: At 4.23 effective bits-per-parameter (BPP), we achieve a perplexity of 14.23, significantly outperforming the standard INT4 baseline (20.03 PPL at 4.71 BPP). (2) High-Fidelity Compression: Relative to the FP16 baseline, HAS-VQ achieves a 2.3x reduction in model size (7.03 BPP) while maintaining statistically indistinguishable perplexity (10.12 vs. 10.04), effectively offering a lossless compression alternative for bandwidth-constrained environments. The code is available at https://github.com/VladimerKhasia/HASVQ