Cloud-Native Vector Search: A Comprehensive Performance Analysis

TL;DR

This paper systematically analyzes cloud-native vector search, comparing cluster and graph indexes, and explores how indexing, search parameters, and caching affect performance in cloud environments.

Contribution

It provides a comprehensive study of index choices, parameter tuning, and caching strategies for cloud-native vector search, highlighting when graph indexes outperform cluster indexes.

Findings

Graph indexes are preferred for high concurrency and high-dimensional data.

Cloud-native search requires different index and parameter configurations than on-disk search.

Certain index optimizations can conflict with caching, affecting performance.

Abstract

Vector search has been widely employed in recommender system and retrieval-augmented-generation pipelines, commonly performed with vector indexes to efficiently find similar items in large datasets. Recent growths in both data and task complexity have motivated placing vector indexes onto remote storage -- cloud-native vector search, which cloud providers have recently introduced services for. Yet, despite varying workload characteristics and various available vector index forms, providers default to using cluster-based indexes, which on paper do adapt well to differences between disk and cloud-based environment: their fetch granularities and lack of notable intra-query dependencies aligns with the large optimal fetch sizes and minimizes costly round-trips (i.e., as opposed to graph-based indexes) to remote storage, respectively. This paper systematically studies cloud-native vector search: What and how should indexes be built and used for on-cloud vector search? We analyze bottlenecks of two common index classes, cluster and graph indexes, on remote storage, and show that despite current standardized adoption of cluster indexes on the cloud, graph indexes are favored in workloads requiring high concurrency and recall, or operating on high-dimensional data or large datatypes. We further find that on-cloud search demands significantly different indexing and search parameterizations versus on-disk search for optimal performance. Finally, we incorporate existing cloud-based caching setups into vector search and find that certain index optimizations work against caching, and study how this can be mitigated to maximize gains under various available cache sizes.