Object localization in ImageNet by looking out of the window

TL;DR

This paper introduces a novel object localization method in ImageNet that considers the relationships between candidate windows, leading to improved accuracy over traditional isolated window scoring techniques.

Contribution

It presents a new scoring approach that incorporates spatial and appearance relations among windows, with an efficient optimization and learning procedure.

Findings

Significantly improves localization accuracy on ImageNet.

Efficient optimization over all candidate windows.

Outperforms recent isolated window scoring methods.

Abstract

We propose a method for annotating the location of objects in ImageNet. Traditionally, this is cast as an image window classification problem, where each window is considered independently and scored based on its appearance alone. Instead, we propose a method which scores each candidate window in the context of all other windows in the image, taking into account their similarity in appearance space as well as their spatial relations in the image plane. We devise a fast and exact procedure to optimize our scoring function over all candidate windows in an image, and we learn its parameters using structured output regression. We demonstrate on 92000 images from ImageNet that this significantly improves localization over recent techniques that score windows in isolation.