MP-RW-LSH: An Efficient Multi-Probe LSH Solution to ANNS in $L_1$ Distance

TL;DR

This paper introduces MP-RW-LSH, the first multi-probe LSH method for approximate nearest neighbor search in L1 distance, significantly reducing the number of hash tables needed compared to previous solutions.

Contribution

It presents MP-RW-LSH, a novel multi-probe LSH scheme for L1 distance, and explains why existing CP-LSH is unsuitable for multi-probe extensions.

Findings

MP-RW-LSH uses 15 to 53 times fewer hash tables than CP-LSH.

Achieves similar query accuracy with fewer hash tables.

First multi-probe LSH solution for ANNS in L1 distance.

Abstract

Approximate Nearest Neighbor Search (ANNS) is a fundamental algorithmic problem, with numerous applications in many areas of computer science. Locality-sensitive hashing (LSH) is one of the most popular solution approaches for ANNS. A common shortcoming of many LSH schemes is that since they probe only a single bucket in a hash table, they need to use a large number of hash tables to achieve a high query accuracy. For ANNS-$L_2$, a multi-probe scheme was proposed to overcome this drawback by strategically probing multiple buckets in a hash table. In this work, we propose MP-RW-LSH, the first and so far only multi-probe LSH solution to ANNS in $L_1$ distance. Another contribution of this work is to explain why a state-of-the-art ANNS-$L_1$ solution called Cauchy projection LSH (CP-LSH) is fundamentally not suitable for multi-probe extension. We show that MP-RW-LSH uses 15 to 53 times fewer hash tables than CP-LSH for achieving similar query accuracies.