Addition is almost all you need: Compressing large language models with double binary factorization

TL;DR

This paper introduces Double Binary Factorization (DBF), a novel binary matrix factorization technique that compresses large language models efficiently, maintaining accuracy while offering flexible compression ratios and competitive performance.

Contribution

The paper presents DBF, a new binary factorization method that improves model compression efficiency and accuracy, with fine-grained control over compression levels and an estimation algorithm for layer-wise ratios.

Findings

DBF outperforms existing binarization methods at 1-bit per weight.

DBF is competitive with state-of-the-art quantization methods at 2-bit per weight.

The proposed layer-wise ratio estimation enhances compression control.

Abstract

Binary quantization approaches, which replace weight matrices with binary matrices and substitute costly multiplications with cheaper additions, offer a computationally efficient approach to address the increasing computational and storage requirements of Large Language Models (LLMs). However, the severe quantization constraint ($\pm1$) can lead to significant accuracy degradation. In this paper, we propose Double Binary Factorization (DBF), a novel method that factorizes dense weight matrices into products of two binary (sign) matrices, each accompanied by scaling vectors. DBF preserves the efficiency advantages of binary representations while achieving compression rates that are competitive with or superior to state-of-the-art methods. Specifically, in a 1-bit per weight range, DBF is better than existing binarization approaches. In a 2-bit per weight range, DBF is competitive with the best quantization methods like QuIP# and QTIP. Unlike most existing compression techniques, which offer limited compression level choices, DBF allows fine-grained control over compression ratios by adjusting the factorization's intermediate dimension. Based on this advantage, we further introduce an algorithm for estimating non-uniform layer-wise compression ratios for DBF, based on previously developed channel pruning criteria. The code is available at: https://github.com/usamec/double_binary