Low-Latency Out-of-Core ANN Search in High-Dimensional Space

TL;DR

SkipDisk is a hybrid disk-memory approach for high-dimensional approximate nearest neighbor search that significantly reduces memory usage while maintaining or improving low-latency search performance.

Contribution

The paper introduces SkipDisk, a novel method combining tight lower bound filtering, data pruning, and asynchronous I/O to optimize disk-memory hybrid ANN search.

Findings

Achieves 85% of HNSW's latency with 10% memory footprint.

Reduces search latency to 63% of HNSW's latency.

Maintains competitive accuracy with significantly less memory.

Abstract

In-memory graph-based approximate nearest neighbor (ANN) search has superior search performance but incurs significant memory footprint. Disk-based methods reduce memory usage but suffer from high disk access latency. A common challenge is how to achieve low-latency search while significantly reducing memory footprint. In this paper, we propose SkipDisk, a disk-memory hybrid ANN search that significantly reduces memory footprint while achieving search latency comparable to or lower than in-memory method HNSW. By analyzing existing disk-based methods, we observed that disk access remains the primary bottleneck, and existing lower bound based filtering methods are two loose to effectively reduce disk access. Therefore, we design SkipDisk to achieve tight lower bound with low memory footprint to reduce the search latency. First, we design a dedicated pivot for each point to improve the lower bound of the triangle inequality for effective filtering. We further design an estimation-based approach based on this lower bound. Second, to reduce the memory footprint, we employ a three-level data pruning strategy to preserve informative data in memory. Third, to further reduce search latency, we design an asynchronous I/O strategy based on the decoupling of in-memory search and disk access by storing neighbor nodes in memory. Experiments show that our method achieves a latency of 85 of HNSW's latency with approximately 10 memory footprint, and a latency to 63 of HNSW's with a slightly higher memory footprint of around 20.