Accelerating Nearest Neighbor Search on Manycore Systems

TL;DR

This paper introduces parallelizable algorithms for metric similarity search optimized for modern multicore and GPU hardware, achieving sublinear search times based on data intrinsic dimensionality.

Contribution

The paper presents novel, hardware-efficient algorithms for metric similarity search that are easily parallelizable and provably sublinear in database size.

Findings

Substantial speedups on various datasets and hardware platforms

Algorithms are simple, parallelizable, and provably sublinear

Effective on multicore CPUs and GPUs

Abstract

We develop methods for accelerating metric similarity search that are effective on modern hardware. Our algorithms factor into easily parallelizable components, making them simple to deploy and efficient on multicore CPUs and GPUs. Despite the simple structure of our algorithms, their search performance is provably sublinear in the size of the database, with a factor dependent only on its intrinsic dimensionality. We demonstrate that our methods provide substantial speedups on a range of datasets and hardware platforms. In particular, we present results on a 48-core server machine, on graphics hardware, and on a multicore desktop.