Approximate Nearest Neighbor Search for Modern AI: A Projection-Augmented Graph Approach

TL;DR

This paper introduces Projection-Augmented Graph (PAG), a novel ANNS framework that improves query efficiency, scalability, and online update support for modern AI workloads by integrating projection techniques into graph indexing.

Contribution

The paper presents PAG, a new ANNS method that combines projection techniques with graph indexing to meet modern AI demands, outperforming existing solutions in speed and robustness.

Findings

PAG achieves up to 5x faster QPS than HNSW.

PAG offers fast indexing and moderate memory footprint.

PAG maintains robustness with increasing dimensionality and retrieval size.

Abstract

Approximate Nearest Neighbor Search (ANNS) is fundamental to modern AI applications. Most existing solutions optimize query efficiency but fail to align with the practical requirements of modern workloads. In this paper, we outline six critical demands of modern AI applications: high query efficiency, fast indexing, low memory footprint, scalability to high dimensionality, robustness across varying retrieval sizes, and support for online insertions. To satisfy all these demands, we introduce Projection-Augmented Graph (PAG), a new ANNS framework that integrates projection techniques into a graph index. PAG reduces unnecessary exact distance computations through asymmetric comparisons between exact and approximate distances as guided by projection-based statistical tests. Three key components are designed and unified to the graph index to optimize indexing and searching. Experiments on six modern datasets demonstrate that PAG consistently achieves superior query per second (QPS)-recall performance -- up to 5x faster than HNSW -- while offering fast indexing speed and moderate memory footprint. PAG remains robust as dimensionality and retrieval size increase and naturally supports online insertions.