Exploiting Local Features from Deep Networks for Image Retrieval

TL;DR

This paper explores how features from different layers of deep CNNs can be used for image retrieval, showing that intermediate layers often outperform the last layers, leading to improved retrieval performance.

Contribution

It introduces a method to extract and encode features from various CNN layers for image retrieval, demonstrating the effectiveness of intermediate layers over the last layer.

Findings

Intermediate CNN layers yield better retrieval results than last layers.

Using VLAD encoding with features from different layers improves performance.

State-of-the-art results achieved with 128-D VLAD descriptors on multiple datasets.

Abstract

Deep convolutional neural networks have been successfully applied to image classification tasks. When these same networks have been applied to image retrieval, the assumption has been made that the last layers would give the best performance, as they do in classification. We show that for instance-level image retrieval, lower layers often perform better than the last layers in convolutional neural networks. We present an approach for extracting convolutional features from different layers of the networks, and adopt VLAD encoding to encode features into a single vector for each image. We investigate the effect of different layers and scales of input images on the performance of convolutional features using the recent deep networks OxfordNet and GoogLeNet. Experiments demonstrate that intermediate layers or higher layers with finer scales produce better results for image retrieval, compared to the last layer. When using compressed 128-D VLAD descriptors, our method obtains state-of-the-art results and outperforms other VLAD and CNN based approaches on two out of three test datasets. Our work provides guidance for transferring deep networks trained on image classification to image retrieval tasks.