Bang for the Buck: Vector Search on Cloud CPUs

TL;DR

This paper evaluates the performance and cost-efficiency of different cloud CPU architectures for vector search workloads, providing benchmarks to guide users in selecting optimal hardware for their needs.

Contribution

It offers the first comprehensive benchmark of vector search performance across various cloud CPU microarchitectures, highlighting cost-performance trade-offs.

Findings

AMD Zen4 outperforms Intel Sapphire Rapids in IVF index queries.

Graviton3 offers the best queries per dollar for most indexes.

Performance varies significantly across CPU architectures depending on the index type.

Abstract

Vector databases have emerged as a new type of systems that support efficient querying of high-dimensional vectors. Many of these offer their database as a service in the cloud. However, the variety of available CPUs and the lack of vector search benchmarks across CPUs make it difficult for users to choose one. In this study, we show that CPU microarchitectures available in the cloud perform significantly differently across vector search scenarios. For instance, in an IVF index on float32 vectors, AMD's Zen4 gives almost 3x more queries per second (QPS) compared to Intel's Sapphire Rapids, but for HNSW indexes, the tables turn. However, when looking at the number of queries per dollar (QP$), Graviton3 is the best option for most indexes and quantization settings, even over Graviton4 (Table 1). With this work, we hope to guide users in getting the best "bang for the buck" when deploying vector search systems.