Randomized embeddings with slack, and high-dimensional Approximate Nearest Neighbor

TL;DR

This paper introduces a novel approach for approximate nearest neighbor search in high-dimensional Euclidean spaces using randomized embeddings and slack, achieving improved query times and space efficiency compared to existing methods like LSH.

Contribution

It proposes a new low-quality embedding framework and a randomized projection technique that reduces dimensionality, leading to more efficient ANN algorithms with better theoretical and practical performance.

Findings

Achieves faster query times than BBD-tree based methods.

Provides linear space data structures for approximate near neighbor search.

Experimental results outperform theoretical predictions up to 500 dimensions.

Abstract

The approximate nearest neighbor problem ($\epsilon$-ANN) in high dimensional Euclidean space has been mainly addressed by Locality Sensitive Hashing (LSH), which has polynomial dependence in the dimension, sublinear query time, but subquadratic space requirement. In this paper, we introduce a new definition of "low-quality" embeddings for metric spaces. It requires that, for some query point $q$, there exists an approximate nearest neighbor among the pre-images of the $k>1$ approximate nearest neighbors in the target space. Focusing on Euclidean spaces, we employ random projections in order to reduce the original problem to one in a space of dimension inversely proportional to $k$. The $k$ approximate nearest neighbors can be efficiently retrieved by a data structure such as BBD-trees. The same approach is applied to the problem of computing an approximate near neighbor, where we obtain a data structure requiring linear space, and query time in $O(d n^{\rho})$, for $\rho\approx 1-\epsilon^2/\log(1/\epsilon)$. This directly implies a solution for $\epsilon$-ANN, while achieving a better exponent in the query time than the method based on BBD-trees. Better bounds are obtained in the case of doubling subsets of $\ell_2$, by combining our method with $r$-nets. We implement our method in C++, and present experimental results in dimension up to $500$ and $10^6$ points, which show that performance is better than predicted by the analysis. In addition, we compare our ANN approach to E2LSH, which implements LSH, and we show that the theoretical advantages of each method are reflected on their actual performance.