RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs

TL;DR

RaBiT introduces a novel residual hierarchy-based binarization framework for large language models, significantly improving accuracy and efficiency in quantization-aware training.

Contribution

It proposes a new method to prevent feature co-adaptation in binary paths by enforcing a residual hierarchy derived from shared weights.

Findings

RaBiT achieves state-of-the-art 2-bit LLM performance.

It delivers a 4.49x inference speed-up on RTX 4090.

RaBiT rivals hardware-intensive vector quantization methods.

Abstract

Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a critical trade-off between low-bit efficiency and performance. Residual binarization enables hardware-friendly, matmul-free inference by stacking binary ($\pm$1) layers, but is plagued by pathological feature co-adaptation. We identify a key failure mode, which we term inter-path adaptation: during quantization-aware training (QAT), parallel residual binary paths learn redundant features, degrading the error-compensation structure and limiting the expressive capacity of the model. While prior work relies on heuristic workarounds (e.g., path freezing) that constrain the solution space, we propose RaBiT, a novel quantization framework that resolves co-adaptation by algorithmically enforcing a residual hierarchy. Its core mechanism sequentially derives each binary path from a single shared full-precision weight, which ensures that every path corrects the error of the preceding one. This process is stabilized by a robust initialization that prioritizes functional preservation over mere weight approximation. RaBiT redefines the 2-bit accuracy-efficiency frontier: it achieves state-of-the-art performance, rivals even hardware-intensive Vector Quantization (VQ) methods, and delivers a $4.49\times$ inference speed-up over full-precision models on an RTX 4090.