VecFlow: A High-Performance Vector Data Management System for Filtered-Search on GPUs

TL;DR

VecFlow is a GPU-optimized vector search system that significantly outperforms CPU-based solutions in throughput and recall, enabling efficient filtered-ANNS for AI applications.

Contribution

The paper introduces VecFlow, a novel GPU-based vector filtered search system with a label-centric index and architecture-aware optimizations, achieving high throughput and recall.

Findings

Achieves 5 million QPS at 90% recall on NVIDIA A100 GPU.

Outperforms CPU solutions like Filtered-DiskANN by up to 135 times.

Supports high recall (99%) with better scalability than existing GPU baselines.

Abstract

Vector search and database systems have become a keystone component in many AI applications. While many prior research has investigated how to accelerate the performance of generic vector search, emerging AI applications require running more sophisticated vector queries efficiently, such as vector search with attribute filters. Unfortunately, recent filtered-ANNS solutions are primarily designed for CPUs, with few exploration and limited performance of filtered-ANNS that take advantage of the massive parallelism offered by GPUs. In this paper, we present VecFlow, a novel high-performance vector filtered search system that achieves unprecedented high throughput and recall while obtaining low latency for filtered-ANNS on GPUs. We propose a novel label-centric indexing and search algorithm that significantly improves the selectivity of ANNS with filters. In addition to algorithmic level optimization, we provide architectural-aware optimization for VecFlow's functional modules, effectively supporting both small batch and large batch queries, and single-label and multi-label query processing. Experimental results on NVIDIA A100 GPU over several public available datasets validate that VecFlow achieves 5 million QPS for recall 90%, outperforming state-of-the-art CPU-based solutions such as Filtered-DiskANN by up to 135 times. Alternatively, VecFlow can easily extend its support to high recall 99% regime, whereas strong GPU-based baselines plateau at around 80% recall. The source code is available at https://github.com/Supercomputing-System-AI-Lab/VecFlow.