AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization

TL;DR

AAAC introduces a lightweight, activation-aware adaptive codebook method for 4-bit LLM weight quantization, significantly improving accuracy with minimal additional computation and no extra memory.

Contribution

The paper proposes AAAC, a novel adaptive codebook approach that uses learned scalar codebooks and activation-weighted error minimization, outperforming existing methods efficiently.

Findings

AAAC completes in 3-30 minutes on a single GPU.

AAAC outperforms baseline methods in accuracy.

AAAC adds no memory overhead beyond the original model.

Abstract

Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference. Existing PTQ methods, such as AWQ and GPTQ, improve how weights are mapped onto a fixed 4-bit grid through scaling, clipping, or error compensation. To further improve accuracy, methods such as OmniQuant and QuIP\# uses gradient-assisted algorithms at the cost of hours of quantization time. In this work, we propose AAAC (Activation-Aware Adaptive Codebooks), a lightweight method for 4-bit LLM weight quantization. AAAC replaces the fixed scalar codebook used in standard quantization with two small learned scalar codebooks (64 bytes) per layer. Each group of weights selects the codebook that minimizes activation-weighted reconstruction error, encoding the choice in the unused sign bit of the group's positive scale and adding zero storage overhead. AAAC completes in 3--30 minutes on a single GPU, and adds no memory beyond the model itself. We evaluate against AWQ, GPTQ, IF4, GPTVQ, OmniQuant, SqueezeLLM, and QuIP\# across model families. AAAC outperforms baselines at orders-of-magnitude less quantization time.