Fast Graph Vector Search via Hardware Acceleration and Delayed-Synchronization Traversal

TL;DR

This paper introduces Falcon, a hardware accelerator, and DST, a traversal algorithm, to significantly reduce latency and improve energy efficiency in graph-based vector search systems used in LLMs and search engines.

Contribution

The paper presents a co-designed hardware and algorithm approach, combining Falcon FPGA accelerator and DST traversal to enhance GVS performance and energy efficiency.

Findings

Up to 4.3x lower latency compared to CPU systems.

Up to 8.0x energy efficiency improvement over GPU systems.

Prototyped on FPGAs, demonstrating significant performance gains.

Abstract

Vector search systems are indispensable in large language model (LLM) serving, search engines, and recommender systems, where minimizing online search latency is essential. Among various algorithms, graph-based vector search (GVS) is particularly popular due to its high search performance and quality. However, reducing GVS latency by intra-query parallelization remains challenging due to limitations imposed by both existing hardware architectures (CPUs and GPUs) and the inherent difficulty of parallelizing graph traversals. To efficiently serve low-latency GVS, we co-design hardware and algorithm by proposing Falcon and Delayed-Synchronization Traversal (DST). Falcon is a hardware GVS accelerator that implements efficient GVS operators, pipelines these operators, and reduces memory accesses by tracking search states with an on-chip Bloom filter. DST is an efficient graph traversal algorithm that simultaneously improves search performance and quality by relaxing traversal orders to maximize accelerator utilization. Evaluation across various graphs and datasets shows that Falcon, prototyped on FPGAs, together with DST, achieves up to 4.3x and 19.5x lower latency and up to 8.0x and 26.9x improvements in energy efficiency over CPU- and GPU-based GVS systems.