Soft Proposal Networks for Weakly Supervised Object Localization

TL;DR

This paper introduces Soft Proposal Networks (SPNs), an end-to-end trainable CNN architecture that efficiently generates object proposals for weakly supervised object localization, significantly improving accuracy and speed.

Contribution

The paper presents the first integration of weakly supervised object proposals into CNNs with an end-to-end learning framework, achieving faster and more accurate localization.

Findings

Achieves state-of-the-art results on PASCAL VOC, MS COCO, and ImageNet.

Object proposals are generated iteratively based on deep features and jointly optimized.

Significantly faster than existing methods, nearly cost-free object proposal generation.

Abstract

Weakly supervised object localization remains challenging, where only image labels instead of bounding boxes are available during training. Object proposal is an effective component in localization, but often computationally expensive and incapable of joint optimization with some of the remaining modules. In this paper, to the best of our knowledge, we for the first time integrate weakly supervised object proposal into convolutional neural networks (CNNs) in an end-to-end learning manner. We design a network component, Soft Proposal (SP), to be plugged into any standard convolutional architecture to introduce the nearly cost-free object proposal, orders of magnitude faster than state-of-the-art methods. In the SP-augmented CNNs, referred to as Soft Proposal Networks (SPNs), iteratively evolved object proposals are generated based on the deep feature maps then projected back, and further jointly optimized with network parameters, with image-level supervision only. Through the unified learning process, SPNs learn better object-centric filters, discover more discriminative visual evidence, and suppress background interference, significantly boosting both weakly supervised object localization and classification performance. We report the best results on popular benchmarks, including PASCAL VOC, MS COCO, and ImageNet.