Graph-based Nearest Neighbor Search: From Practice to Theory

TL;DR

This paper provides a rigorous theoretical analysis of graph-based nearest neighbor search algorithms, especially in low-dimensional settings, complementing their empirical success with formal guarantees.

Contribution

It offers the first theoretical guarantees for graph-based NNS algorithms, analyzing basic and heuristic methods in low-dimensional regimes.

Findings

Theoretical guarantees for greedy graph-based NNS algorithms.

Analysis of heuristics like shortcut edges and dynamic candidate lists.

Experimental validation supporting the theoretical insights.

Abstract

Graph-based approaches are empirically shown to be very successful for the nearest neighbor search (NNS). However, there has been very little research on their theoretical guarantees. We fill this gap and rigorously analyze the performance of graph-based NNS algorithms, specifically focusing on the low-dimensional (d << \log n) regime. In addition to the basic greedy algorithm on nearest neighbor graphs, we also analyze the most successful heuristics commonly used in practice: speeding up via adding shortcut edges and improving accuracy via maintaining a dynamic list of candidates. We believe that our theoretical insights supported by experimental analysis are an important step towards understanding the limits and benefits of graph-based NNS algorithms.