Towards Efficient and Scalable Distributed Vector Search with RDMA

TL;DR

CoTra is a distributed vector search system that improves scalability and efficiency by combining algorithm-system co-design, asynchronous execution, and system optimizations, achieving significant speedups over single-machine and baseline methods.

Contribution

The paper introduces CoTra, a novel system for distributed vector search that effectively balances computation and communication through innovative co-design and system optimizations.

Findings

CoTra achieves 9.8-13.4x query throughput scaling on 16 machines.

It outperforms baseline methods by 2.12-3.58x at 0.95 recall@10.

System optimizations significantly improve distributed search efficiency.

Abstract

Similarity-based vector search facilitates many important applications such as search and recommendation but is limited by the memory capacity and bandwidth of a single machine due to large datasets and intensive data read. In this paper, we present CoTra, a system that scales up vector search for distributed execution. We observe a tension between computation and communication efficiency, which is the main challenge for good scalability, i.e., handling the local vectors on each machine independently blows up computation as the pruning power of vector index is not fully utilized, while running a global index over all machines introduces rich data dependencies and thus extensive communication. To resolve such tension, we leverage the fact that vector search is approximate in nature and robust to asynchronous execution. In particular, we run collaborative vector search over the machines with algorithm-system co-designs including clustering-based data partitioning to reduce communication, asynchronous execution to avoid communication stall, and task push to reduce network traffic. To make collaborative search efficient, we introduce a suite of system optimizations including task scheduling, communication batching, and storage format. We evaluate CoTra on real datasets and compare with four baselines. The results show that when using 16 machines, the query throughput of CoTra scales to 9.8-13.4x over a single machine and is 2.12-3.58x of the best-performing baseline at 0.95 recall@10.