PathWeaver: A High-Throughput Multi-GPU System for Graph-Based Approximate Nearest Neighbor Search

TL;DR

PathWeaver is a multi-GPU system that significantly accelerates graph-based approximate nearest neighbor search by introducing pipelining, ghost staging, and direction-guided selection, achieving over threefold speedup on large datasets.

Contribution

The paper introduces PathWeaver, a novel multi-GPU framework with new techniques to improve scalability and efficiency of ANNS on large datasets.

Findings

Achieves 3.24× average speedup over state-of-the-art methods.

Up to 5.30× speedup at 95% recall rate.

Effectively scales to large datasets with improved GPU utilization.

Abstract

Graph-based Approximate Nearest Neighbor Search (ANNS) is widely adopted in numerous applications, such as recommendation systems, natural language processing, and computer vision. While recent works on GPU-based acceleration have significantly advanced ANNS performance, the ever-growing scale of datasets now demands efficient multi-GPU solutions. However, the design of existing works overlooks multi-GPU scalability, resulting in naive approaches that treat additional GPUs as a means to extend memory capacity for large datasets. This inefficiency arises from partitioning the dataset and independently searching for data points similar to the queries in each GPU. We therefore propose PathWeaver, a novel multi-GPU framework designed to scale and accelerate ANNS for large datasets. First, we propose pipelining-based path extension, a GPU-aware pipelining mechanism that reduces prior work's redundant search iterations by leveraging GPU-to-GPU communication. Second, we design ghost staging that leverages a representative dataset to identify optimal query starting points, reducing the search space for challenging queries. Finally, we introduce direction-guided selection, a data selection technique that filters irrelevant points early in the search process, minimizing unnecessary memory accesses and distance computations. Comprehensive evaluations across diverse datasets demonstrate that PathWeaver achieves 3.24$\times$ geomean speedup and up to 5.30$\times$ speedup on 95% recall rate over state-of-the-art multi-GPU-based ANNS frameworks.