KBest: Efficient Vector Search on Kunpeng CPU

TL;DR

KBest is a hardware-aware vector search library optimized for Huawei Kunpeng CPUs, achieving over 2x throughput improvements by leveraging SIMD, prefetching, and quantization, thus enabling efficient large-scale vector search applications.

Contribution

The paper introduces KBest, a novel vector search library specifically optimized for ARM-based Kunpeng CPUs, with extensive hardware-aware and algorithmic enhancements.

Findings

KBest outperforms existing x86-optimized libraries in throughput.

Optimizations lead to over 2x increase in query processing speed.

KBest handles tens of millions of queries daily for real-world applications.

Abstract

Vector search, which returns the vectors most similar to a given query vector from a large vector dataset, underlies many important applications such as search, recommendation, and LLMs. To be economic, vector search needs to be efficient to reduce the resources required by a given query workload. However, existing vector search libraries (e.g., Faiss and DiskANN) are optimized for x86 CPU architectures (i.e., Intel and AMD CPUs) while Huawei Kunpeng CPUs are based on the ARM architecture and competitive in compute power. In this paper, we present KBest as a vector search library tailored for the latest Kunpeng 920 CPUs. To be efficient, KBest incorporates extensive hardware-aware and algorithmic optimizations, which include single-instruction-multiple-data (SIMD) accelerated distance computation, data prefetch, index refinement, early termination, and vector quantization. Experiment results show that KBest outperforms SOTA vector search libraries running on x86 CPUs, and our optimizations can improve the query throughput by over 2x. Currently, KBest serves applications from both our internal business and external enterprise clients with tens of millions of queries on a daily basis.