Hashing as Tie-Aware Learning to Rank

TL;DR

This paper introduces tie-aware ranking metrics for hashing, enabling direct optimization of retrieval performance measures like AP and NDCG using deep neural networks, leading to state-of-the-art results in image retrieval.

Contribution

It develops tie-aware evaluation metrics and continuous relaxations for optimizing ranking-based hashing, a novel approach for improving retrieval quality.

Findings

Achieves new state-of-the-art in image retrieval benchmarks.

Demonstrates the effectiveness of tie-aware metrics in hashing.

Provides a gradient-based optimization framework for ranking metrics.

Abstract

Hashing, or learning binary embeddings of data, is frequently used in nearest neighbor retrieval. In this paper, we develop learning to rank formulations for hashing, aimed at directly optimizing ranking-based evaluation metrics such as Average Precision (AP) and Normalized Discounted Cumulative Gain (NDCG). We first observe that the integer-valued Hamming distance often leads to tied rankings, and propose to use tie-aware versions of AP and NDCG to evaluate hashing for retrieval. Then, to optimize tie-aware ranking metrics, we derive their continuous relaxations, and perform gradient-based optimization with deep neural networks. Our results establish the new state-of-the-art for image retrieval by Hamming ranking in common benchmarks.