Block-Sphere Vector Quantization

TL;DR

This paper compares existing rotation-based vector quantizers theoretically, introduces a new spherical block quantizer called BlockQuant, and demonstrates its practical advantages for embedding compression and language model inference.

Contribution

It provides a unified theoretical comparison of rotation-based quantizers and introduces BlockQuant, a novel spherical block quantizer with improved theoretical and practical performance.

Findings

Unified comparison clarifies criterion-dependent advantages of existing methods.

BlockQuant preserves spherical geometry, improving reconstruction and inner-product distortion.

Experiments show practical gains in embedding compression and language model tasks.

Abstract

Vector quantization is a fundamental primitive for scalable machine learning systems, enabling memory-efficient storage, fast retrieval, and compressed inference. Recent rotation-based quantizers such as EDEN, RabitQ, and TurboQuant have introduced strong guarantees and empirical performance, but the surrounding comparisons have been difficult to interpret because they rely on different distortion criteria, probability regimes, and implementation assumptions. As our first contribution, we provide a unified theoretical comparison of these methods and show that their relative advantages are criterion-dependent rather than absolute: EDEN and TurboQuant are favorable for MSE distortion, EDEN is also effective for expected inner-product distortion, and RabitQ provides strong high-probability control. This comparison further clarifies that EDEN provides particularly strong guarantees for expected distortion measures. As our second contribution, we introduce Block-Sphere Quantization (BlockQuant), a new rotation-based block quantization algorithm designed around the spherical geometry of randomly rotated vectors. Unlike coordinate-wise quantizers, BlockQuant quantizes blocks on the sphere, preserving the geometry of rotated embeddings more faithfully. We prove that this block-spherical design theoretically improves over the baselines considered in this paper for both reconstruction MSE and expected inner-product distortion. Our experiments on real embedding datasets and long-context LLM inference tasks show practical gains that are consistent with our theoretical improvements.