Box-based Refinement for Weakly Supervised and Unsupervised Localization Tasks

TL;DR

This paper introduces a box-based refinement method that enhances weakly supervised and unsupervised localization tasks by training detectors on network outputs, leading to significant performance improvements.

Contribution

The paper proposes a novel box-based refinement approach that trains detectors on network outputs rather than raw images, improving localization performance in weakly supervised and unsupervised tasks.

Findings

Improved phrase grounding performance.

Enhanced unsupervised object discovery.

Detectors trained on network outputs yield better localization.

Abstract

It has been established that training a box-based detector network can enhance the localization performance of weakly supervised and unsupervised methods. Moreover, we extend this understanding by demonstrating that these detectors can be utilized to improve the original network, paving the way for further advancements. To accomplish this, we train the detectors on top of the network output instead of the image data and apply suitable loss backpropagation. Our findings reveal a significant improvement in phrase grounding for the ``what is where by looking'' task, as well as various methods of unsupervised object discovery. Our code is available at https://github.com/eyalgomel/box-based-refinement.