Fast Spectral Ranking for Similarity Search

TL;DR

This paper introduces a fast spectral ranking method that reduces manifold search to Euclidean search, significantly improving object retrieval efficiency and accuracy on large-scale datasets.

Contribution

It proposes an explicit embedding using approximate Fourier basis to enable fast similarity search on high-dimensional manifold representations.

Findings

Improves state-of-the-art on object retrieval datasets including Instre.

Achieves fast query times comparable to standard similarity search.

Offline computation takes only a few hours at scale of 10^5 images.

Abstract

Despite the success of deep learning on representing images for particular object retrieval, recent studies show that the learned representations still lie on manifolds in a high dimensional space. This makes the Euclidean nearest neighbor search biased for this task. Exploring the manifolds online remains expensive even if a nearest neighbor graph has been computed offline. This work introduces an explicit embedding reducing manifold search to Euclidean search followed by dot product similarity search. This is equivalent to linear graph filtering of a sparse signal in the frequency domain. To speed up online search, we compute an approximate Fourier basis of the graph offline. We improve the state of art on particular object retrieval datasets including the challenging Instre dataset containing small objects. At a scale of 10^5 images, the offline cost is only a few hours, while query time is comparable to standard similarity search.