Deep Convolutional Features for Image Based Retrieval and Scene Categorization

TL;DR

This paper explores alternative pooling strategies for CNN features, focusing on earlier layers to improve image retrieval and scene categorization efficiency, and introduces a new dataset for geographically diverse images.

Contribution

It proposes a novel pooling approach using features from an earlier CNN layer, enhancing retrieval and categorization performance with lower computational costs.

Findings

Superior retrieval performance on INRIA Holidays dataset

Effective scene categorization on SUN397 dataset

Introduction of GeoPlaces5K dataset for diverse geographical images

Abstract

Several recent approaches showed how the representations learned by Convolutional Neural Networks can be repurposed for novel tasks. Most commonly it has been shown that the activation features of the last fully connected layers (fc7 or fc6) of the network, followed by a linear classifier outperform the state-of-the-art on several recognition challenge datasets. Instead of recognition, this paper focuses on the image retrieval problem and proposes a examines alternative pooling strategies derived for CNN features. The presented scheme uses the features maps from an earlier layer 5 of the CNN architecture, which has been shown to preserve coarse spatial information and is semantically meaningful. We examine several pooling strategies and demonstrate superior performance on the image retrieval task (INRIA Holidays) at the fraction of the computational cost, while using a relatively small memory requirements. In addition to retrieval, we see similar efficiency gains on the SUN397 scene categorization dataset, demonstrating wide applicability of this simple strategy. We also introduce and evaluate a novel GeoPlaces5K dataset from different geographical locations in the world for image retrieval that stresses more dramatic changes in appearance and viewpoint.