OrchANN: A Unified I/O Orchestration Framework for Skewed Out-of-Core Vector Search

TL;DR

OrchANN is a novel out-of-core vector search system that improves performance and efficiency by using a unified I/O orchestration model, adaptive indexing, and multi-level pruning to handle skewed workloads at scale.

Contribution

It introduces a unified I/O orchestration framework with adaptive local indexing and multi-level pruning for skewed out-of-core vector search workloads.

Findings

Outperforms existing systems in QPS and latency.

Reduces SSD accesses significantly.

Achieves up to 17.2x higher QPS and 25.0x lower latency.

Abstract

Approximate nearest neighbor search (ANNS) at billion scale is fundamentally an out-of-core problem: vectors and indexes live on SSD, so performance is dominated by I/O rather than compute. Under skewed semantic embeddings, existing out-of-core systems break down: a uniform local index mismatches cluster scales, static routing misguides queries and inflates the number of probed partitions, and pruning is incomplete at the cluster level and lossy at the vector level, triggering "fetch-to-discard" reranking on raw vectors. We present OrchANN, an out-of-core ANNS engine that uses an I/O orchestration model for unified I/O governance along the route-access-verify pipeline. OrchANN selects a heterogeneous local index per cluster via offline auto-profiling, maintains a query-aware in-memory navigation graph that adapts to skewed workloads, and applies multi-level pruning with geometric bounds to filter both clusters and vectors before issuing SSD reads. Across five standard datasets under strict out-of-core constraints, OrchANN outperforms four baselines including DiskANN, Starling, SPANN, and PipeANN in both QPS and latency while reducing SSD accesses. Furthermore, OrchANN delivers up to 17.2x higher QPS and 25.0x lower latency than competing systems without sacrificing accuracy.