CAPS: A Practical Partition Index for Filtered Similarity Search

TL;DR

This paper introduces CAPS, a space partitioning-based index for constrained approximate nearest neighbor search that outperforms graph-based methods in recall-latency tradeoffs with smaller index size.

Contribution

The paper presents CAPS, a novel partition index for constrained ANNS that offers better performance and smaller size compared to existing graph-based algorithms.

Findings

CAPS outperforms state-of-the-art graph-based methods in recall-latency tradeoffs.

CAPS achieves similar or better performance with only 10% of the index size.

The approach demonstrates the effectiveness of space partitioning in constrained ANNS.

Abstract

With the surging popularity of approximate near-neighbor search (ANNS), driven by advances in neural representation learning, the ability to serve queries accompanied by a set of constraints has become an area of intense interest. While the community has recently proposed several algorithms for constrained ANNS, almost all of these methods focus on integration with graph-based indexes, the predominant class of algorithms achieving state-of-the-art performance in latency-recall tradeoffs. In this work, we take a different approach and focus on developing a constrained ANNS algorithm via space partitioning as opposed to graphs. To that end, we introduce Constrained Approximate Partitioned Search (CAPS), an index for ANNS with filters via space partitions that not only retains the benefits of a partition-based algorithm but also outperforms state-of-the-art graph-based constrained search techniques in recall-latency tradeoffs, with only 10% of the index size.