Graph-Based Vector Search: An Experimental Evaluation of the State-of-the-Art

TL;DR

This paper provides an extensive experimental comparison of graph-based vector search methods, analyzing their strengths, limitations, and scalability on large datasets, to guide future research and application.

Contribution

It offers the first comprehensive evaluation of twelve state-of-the-art graph-based vector search algorithms on billion-scale datasets, highlighting key design paradigms and open research directions.

Findings

Incremental insertion and neighborhood diversification are most effective.

Choice of base graph impacts scalability significantly.

Best methods perform well up to 1 billion vectors.

Abstract

Vector data is prevalent across business and scientific applications, and its popularity is growing with the proliferation of learned embeddings. Vector data collections often reach billions of vectors with thousands of dimensions, thus, increasing the complexity of their analysis. Vector search is the backbone of many critical analytical tasks, and graph-based methods have become the best choice for analytical tasks that do not require guarantees on the quality of the answers. We briefly survey in-memory graph-based vector search, outline the chronology of the different methods and classify them according to five main design paradigms: seed selection, incremental insertion, neighborhood propagation, neighborhood diversification, and divide-and-conquer. We conduct an exhaustive experimental evaluation of twelve state-of-the-art methods on seven real data collections, with sizes up to 1 billion vectors. We share key insights about the strengths and limitations of these methods; e.g., the best approaches are typically based on incremental insertion and neighborhood diversification, and the choice of the base graph can hurt scalability. Finally, we discuss open research directions, such as the importance of devising more sophisticated data-adaptive seed selection and diversification strategies.