Embedding Compression via Spherical Coordinates

TL;DR

This paper introduces a novel embedding compression technique using spherical coordinates that achieves 1.5x compression with minimal loss, outperforming previous lossless methods across various data types.

Contribution

The method leverages spherical coordinate properties of high-dimensional vectors to enable efficient entropy coding, achieving superior compression with negligible reconstruction error.

Findings

Achieves 1.5x compression over prior methods.

Maintains zero measurable retrieval degradation.

Consistent performance across diverse embedding types.

Abstract

We present an $\epsilon$-bounded compression method for unit-norm embeddings that achieves 1.5$\times$ compression, 25% better than the best prior lossless method. The method exploits that spherical coordinates of high-dimensional unit vectors concentrate around $\pi/2$, causing IEEE 754 exponents to collapse to a single value and high-order mantissa bits to become predictable, enabling entropy coding of both. Reconstruction error is bounded by float32 machine epsilon ($1.19 \times 10^{-7}$), making reconstructed values indistinguishable from originals at float32 precision. Evaluation across 26 configurations spanning text, image, and multi-vector embeddings confirms consistent compression improvement with zero measurable retrieval degradation on BEIR benchmarks.