Deep Feature Aggregation and Image Re-ranking with Heat Diffusion for Image Retrieval

TL;DR

This paper introduces a novel heat diffusion-based method for deep feature aggregation and image re-ranking in image retrieval, effectively reducing the dominance of repetitive features and improving retrieval accuracy.

Contribution

It presents an unsupervised heat diffusion approach for deep feature aggregation and a re-ranking method based on heat sources, enhancing image retrieval performance.

Findings

Outperforms previous methods on public benchmarks

Effectively reduces over-representation of bursty features

Improves image retrieval accuracy with deep features

Abstract

Image retrieval based on deep convolutional features has demonstrated state-of-the-art performance in popular benchmarks. In this paper, we present a unified solution to address deep convolutional feature aggregation and image re-ranking by simulating the dynamics of heat diffusion. A distinctive problem in image retrieval is that repetitive or \emph{bursty} features tend to dominate final image representations, resulting in representations less distinguishable. We show that by considering each deep feature as a heat source, our unsupervised aggregation method is able to avoid over-representation of \emph{bursty} features. We additionally provide a practical solution for the proposed aggregation method and further show the efficiency of our method in experimental evaluation. Inspired by the aforementioned deep feature aggregation method, we also propose a method to re-rank a number of top ranked images for a given query image by considering the query as the heat source. Finally, we extensively evaluate the proposed approach with pre-trained and fine-tuned deep networks on common public benchmarks and show superior performance compared to previous work.