SpANNS: Optimizing Approximate Nearest Neighbor Search for Sparse Vectors Using Near Memory Processing

TL;DR

SpANNS introduces a near-memory processing architecture that significantly accelerates sparse approximate nearest neighbor search, addressing scalability issues in hybrid retrieval systems by leveraging specialized hardware close to memory.

Contribution

The paper presents SpANNS, a novel near-memory processing architecture that enhances sparse ANNS performance using a hybrid index and runtime optimizations on a CXL platform.

Findings

Achieves 15.2x to 21.6x faster execution than CPU baselines.

Enables scalable and efficient sparse vector search.

Addresses limitations of CPU-based sparse ANNS implementations.

Abstract

Approximate Nearest Neighbor Search (ANNS) is a fundamental operation in vector databases, enabling efficient similarity search in high-dimensional spaces. While dense ANNS has been optimized using specialized hardware accelerators, sparse ANNS remains limited by CPU-based implementations, hindering scalability. This limitation is increasingly critical as hybrid retrieval systems, combining sparse and dense embeddings, become standard in Information Retrieval (IR) pipelines. We propose SpANNS, a near-memory processing architecture for sparse ANNS. SpANNS combines a hybrid inverted index with efficient query management and runtime optimizations. The architecture is built on a CXL Type-2 near-memory platform, where a specialized controller manages query parsing and cluster filtering, while compute-enabled DIMMs perform index traversal and distance computations close to the data. It achieves 15.2x to 21.6x faster execution over the state-of-the-art CPU baselines, offering scalable and efficient solutions for sparse vector search.