From HNSW to Information-Theoretic Binarization: Rethinking the Architecture of Scalable Vector Search

TL;DR

This paper proposes an information-theoretic binary vector search architecture that offers comparable accuracy to traditional high-precision methods but with lower latency, cost, and better scalability for semantic search systems.

Contribution

It introduces a novel binarization and scoring framework that enables exhaustive binary search, eliminating accuracy loss and reducing operational costs compared to existing ANN architectures.

Findings

Comparable retrieval quality to full-precision systems

Significantly lower latency and higher throughput at scale

Enables serverless, cost-effective deployment models

Abstract

Modern semantic search and retrieval-augmented generation (RAG) systems rely predominantly on in-memory approximate nearest neighbor (ANN) indexes over high-precision floating-point vectors, resulting in escalating operational cost and inherent trade-offs between latency, throughput, and retrieval accuracy. This paper analyzes the architectural limitations of the dominant "HNSW + float32 + cosine similarity" stack and evaluates existing cost-reduction strategies, including storage disaggregation and lossy vector quantization, which inevitably sacrifice either performance or accuracy. We introduce and empirically evaluate an alternative information-theoretic architecture based on maximally informative binarization (MIB), efficient bitwise distance metrics, and an information-theoretic scoring (ITS) mechanism. Unlike conventional ANN systems, this approach enables exhaustive search over compact binary representations, allowing deterministic retrieval and eliminating accuracy degradation under high query concurrency. Using the MAIR benchmark across 14 datasets and 10,038 queries, we compare this architecture against Elasticsearch, Pinecone, PGVector, and Qdrant. Results demonstrate retrieval quality comparable to full-precision systems, while achieving substantially lower latency and maintaining constant throughput at high request rates. We show that this architectural shift enables a truly serverless, cost-per-query deployment model, challenging the necessity of large in-memory ANN indexes for high-quality semantic search.