LoRANN: Low-Rank Matrix Factorization for Approximate Nearest Neighbor Search

TL;DR

LoRANN introduces a low-rank matrix factorization approach for approximate nearest neighbor search, improving query speed and memory efficiency over traditional methods, and is competitive with leading graph-based algorithms.

Contribution

The paper presents a novel supervised score computation method using reduced-rank regression for clustering-based ANN, and introduces LoRANN, a new library leveraging this technique.

Findings

RRR outperforms PQ in query latency and memory usage on high-dimensional data.

LoRANN is competitive with top graph-based ANN algorithms.

LoRANN surpasses state-of-the-art GPU ANN methods on high-dimensional datasets.

Abstract

Approximate nearest neighbor (ANN) search is a key component in many modern machine learning pipelines; recent use cases include retrieval-augmented generation (RAG) and vector databases. Clustering-based ANN algorithms, that use score computation methods based on product quantization (PQ), are often used in industrial-scale applications due to their scalability and suitability for distributed and disk-based implementations. However, they have slower query times than the leading graph-based ANN algorithms. In this work, we propose a new supervised score computation method based on the observation that inner product approximation is a multivariate (multi-output) regression problem that can be solved efficiently by reduced-rank regression. Our experiments show that on modern high-dimensional data sets, the proposed reduced-rank regression (RRR) method is superior to PQ in both query latency and memory usage. We also introduce LoRANN, a clustering-based ANN library that leverages the proposed score computation method. LoRANN is competitive with the leading graph-based algorithms and outperforms the state-of-the-art GPU ANN methods on high-dimensional data sets.