GPU-Accelerated ANNS: Quantized for Speed, Built for Change

TL;DR

Jasper is a GPU-native ANNS system that significantly improves query throughput, supports fast dataset updates, and reduces memory usage through novel algorithms and quantization techniques, addressing key limitations of existing GPU-based solutions.

Contribution

Jasper introduces a lock-free batch construction algorithm, an efficient quantization method, and an optimized search kernel, enabling high-performance, updatable, and memory-efficient GPU-accelerated ANNS.

Findings

Jasper achieves up to 1.93x higher query throughput than CAGRA.

Jasper constructs indices 2.4x faster than CAGRA.

Jasper delivers 19-131x faster queries compared to BANG.

Abstract

Approximate nearest neighbor search (ANNS) is a core problem in machine learning and information retrieval applications. GPUs offer a promising path to high-performance ANNS: they provide massive parallelism for distance computations, are readily available, and can co-locate with downstream applications. Despite these advantages, current GPU-accelerated ANNS systems face three key limitations. First, real-world applications operate on evolving datasets that require fast batch updates, yet most GPU indices must be rebuilt from scratch when new data arrives. Second, high-dimensional vectors strain memory bandwidth, but current GPU systems lack efficient quantization techniques that reduce data movement without introducing costly random memory accesses. Third, the data-dependent memory accesses inherent to greedy search make overlapping compute and memory difficult, leading to reduced performance. We present Jasper, a GPU-native ANNS system with both high query throughput and updatability. Jasper builds on the Vamana graph index and overcomes existing bottlenecks via three contributions: (1) a CUDA batch-parallel construction algorithm that enables lock-free streaming insertions, (2) a GPU-efficient implementation of RaBitQ quantization that reduces memory footprint up to 8x without the random access penalties, and (3) an optimized greedy search kernel that increases compute utilization, resulting in better latency hiding and higher throughput. Our evaluation across five datasets shows that Jasper achieves up to 1.93x higher query throughput than CAGRA and achieves up to 80% peak utilization as measured by the roofline model. Jasper's construction scales efficiently and constructs indices an average of 2.4x faster than CAGRA while providing updatability that CAGRA lacks. Compared to BANG, the previous fastest GPU Vamana implementation, Jasper delivers 19-131x faster queries.