Efficient Approximate Nearest Neighbor Search under Multi-Attribute Range Filter

TL;DR

This paper introduces KHI, a novel multi-attribute RFANNS index that combines attribute-space partitioning with HNSW graphs, significantly improving query throughput and recall in high-dimensional nearest neighbor searches with attribute constraints.

Contribution

KHI is the first multi-attribute RFANNS index that effectively combines tree partitioning with HNSW graphs, enabling efficient high-dimensional range-filtered nearest neighbor searches.

Findings

KHI achieves 2.46x higher QPS on average compared to baseline.

KHI improves performance up to 16.22x on challenging datasets.

KHI maintains high recall while providing high throughput.

Abstract

Nearest neighbor search on high-dimensional vectors is fundamental in modern AI and database systems. In many real-world applications, queries involve constraints on multiple numeric attributes, giving rise to range-filtering approximate nearest neighbor search (RFANNS). While there exist RFANNS indexes for single-attribute range predicates, extending them to the multi-attribute setting is nontrivial and often ineffective. In this paper, we propose KHI, an index for multi-attribute RFANNS that combines an attribute-space partitioning tree with HNSW graphs attached to tree nodes. A skew-aware splitting rule bounds the tree height by $O(\log n)$, and queries are answered by routing through the tree and running greedy search on the HNSW graphs. Experiments on four real-world datasets show that KHI consistently achieves high query throughput while maintaining high recall. Compared with the state-of-the-art RFANNS baseline, KHI improves QPS by $2.46\times$ on average and up to $16.22\times$ on the hard dataset, with larger gains for smaller selectivity, larger $k$, and higher predicate cardinality.