More Than Bits: Multi-Envelope Double Binary Factorization for Extreme Quantization

TL;DR

This paper introduces Multi-envelope Double Binary Factorization (MDBF), a novel low-bit quantization method for large language models that improves performance by enhancing magnitude expressiveness while maintaining efficient inference.

Contribution

MDBF extends DBF by using a rank-l envelope with shared sign bases, enabling better performance in extreme quantization of LLMs.

Findings

MDBF improves perplexity over previous binary formats.

MDBF enhances zero-shot accuracy at matched bits per weight.

MDBF maintains deployment-friendly inference primitives.

Abstract

For extreme low-bit quantization of large language models (LLMs), Double Binary Factorization (DBF) is attractive as it enables efficient inference without sacrificing accuracy. However, the scaling parameters of DBF are too restrictive; after factoring out signs, all rank components share the same magnitude profile, resulting in performance saturation. We propose Multi-envelope DBF (MDBF), which retains a shared pair of 1-bit sign bases but replaces the single envelope with a rank-$l$ envelope. By sharing sign matrices among envelope components, MDBF effectively maintains a binary carrier and utilizes the limited memory budget for magnitude expressiveness. We also introduce a closed-form initialization and an alternating refinement method to optimize MDBF. Across the LLaMA and Qwen families, MDBF enhances perplexity and zero-shot accuracy over previous binary formats at matched bits per weight while preserving the same deployment-friendly inference primitive.