Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Models

TL;DR

This paper introduces UniQuanF, a novel quantization method that combines the strengths of uniform and binary-coding schemes to improve the accuracy of large language model compression without additional deployment costs.

Contribution

UniQuanF unifies uniform and binary-coding quantization techniques through flexible mapping, enabling more accurate LLM compression with optimized parameters and theoretical guarantees.

Findings

Achieves up to 4.60% higher accuracy on GSM8K benchmark.

Outperforms existing UQ and BCQ methods.

Removes computational and memory overhead through unification theorem.

Abstract

How can we quantize large language models while preserving accuracy? Quantization is essential for deploying large language models (LLMs) efficiently. Binary-coding quantization (BCQ) and uniform quantization (UQ) are promising quantization schemes that have strong expressiveness and optimizability, respectively. However, neither scheme leverages both advantages. In this paper, we propose UniQuanF (Unified Quantization with Flexible Mapping), an accurate quantization method for LLMs. UniQuanF harnesses both strong expressiveness and optimizability by unifying the flexible mapping technique in UQ and non-uniform quantization levels of BCQ. We propose unified initialization, and local and periodic mapping techniques to optimize the parameters in UniQuanF precisely. After optimization, our unification theorem removes computational and memory overhead, allowing us to utilize the superior accuracy of UniQuanF without extra deployment costs induced by the unification. Experimental results demonstrate that UniQuanF outperforms existing UQ and BCQ methods, achieving up to 4.60% higher accuracy on GSM8K benchmark.