Discriminative Sampling of Proposals in Self-Supervised Transformers for Weakly Supervised Object Localization

TL;DR

This paper introduces a novel discriminative proposal sampling method leveraging self-supervised transformers and CNN classifiers to improve weakly supervised object localization, demonstrating superior results on drone and bird datasets.

Contribution

The paper proposes DiPS, a new approach that uses multiple transformer heads and CNN-based discriminative sampling to enhance WSOL performance.

Findings

Outperforms state-of-the-art on TelDrone dataset

Effective on CUB dataset for different tasks

Improves object localization accuracy

Abstract

Drones are employed in a growing number of visual recognition applications. A recent development in cell tower inspection is drone-based asset surveillance, where the autonomous flight of a drone is guided by localizing objects of interest in successive aerial images. In this paper, we propose a method to train deep weakly-supervised object localization (WSOL) models based only on image-class labels to locate object with high confidence. To train our localizer, pseudo labels are efficiently harvested from a self-supervised vision transformers (SSTs). However, since SSTs decompose the scene into multiple maps containing various object parts, and do not rely on any explicit supervisory signal, they cannot distinguish between the object of interest and other objects, as required WSOL. To address this issue, we propose leveraging the multiple maps generated by the different transformer heads to acquire pseudo-labels for training a deep WSOL model. In particular, a new Discriminative Proposals Sampling (DiPS) method is introduced that relies on a CNN classifier to identify discriminative regions. Then, foreground and background pixels are sampled from these regions in order to train a WSOL model for generating activation maps that can accurately localize objects belonging to a specific class. Empirical results on the challenging TelDrone dataset indicate that our proposed approach can outperform state-of-art methods over a wide range of threshold values over produced maps. We also computed results on CUB dataset, showing that our method can be adapted for other tasks.