Class Correlation affects Single Object Localization using Pre-trained ConvNets

TL;DR

This paper investigates how pre-trained ConvNets can be used for object localization without fine-tuning, revealing that class correlation influences the network's focus on object features.

Contribution

It introduces the EISS algorithm for localizing objects using pre-trained ConvNets and uncovers the impact of class correlation on feature responsiveness.

Findings

ConvNets respond more to object features than descriptors

Class correlation affects the network's focus on features

EISS effectively localizes objects without fine-tuning

Abstract

The problem of object localization has become one of the mainstream problems of vision. Most of the algorithms proposed involve the design for the model to be specifically for localizing objects. In this paper, we explore whether a pre-trained canonical ConvNet (without fine-tuning) trained purely for object classification on one dataset with global image level labels can be used to localize objects in images containing a single instance on a separate dataset while generalizing to novel classes. We propose a simple algorithm involving cropping and blackening out regions in the image space called Explicit Image Space based Search (EISS) for locating the most responsive regions in an image in the context of object localization. EISS brings to light the interesting phenomenon of a ConvNets responding more to features within objects as opposed to object level descriptors, as the classes in the training data get more correlated (visually/semantically similar).