NearBucket-LSH: Efficient Similarity Search in P2P Networks

TL;DR

NearBucket-LSH is a novel algorithm that enhances similarity search in large-scale P2P social networks by balancing search quality and network cost through an innovative extension of Locality Sensitive Hashing.

Contribution

It introduces NearBucket-LSH, an extension of LSH that improves search quality while reducing network costs in distributed social networks.

Findings

Search quality increases by over 50% in many cases.

NearBucket-LSH reduces network cost compared to previous methods.

The algorithm effectively balances search accuracy and communication overhead.

Abstract

We present NearBucket-LSH, an effective algorithm for similarity search in large-scale distributed online social networks organized as peer-to-peer overlays. As communication is a dominant consideration in distributed systems, we focus on minimizing the network cost while guaranteeing good search quality. Our algorithm is based on Locality Sensitive Hashing (LSH), which limits the search to collections of objects, called buckets, that have a high probability to be similar to the query. More specifically, NearBucket-LSH employs an LSH extension that searches in near buckets, and improves search quality but also significantly increases the network cost. We decrease the network cost by considering the internals of both LSH and the P2P overlay, and harnessing their properties to our needs. We show that our NearBucket-LSH increases search quality for a given network cost compared to previous art. In many cases, the search quality increases by more than 50%.