Open-World Weakly-Supervised Object Localization

TL;DR

This paper introduces the OWSOL task for open-world weakly-supervised object localization, proposing a contrastive co-learning method that effectively localizes objects of both known and novel categories without bounding box annotations.

Contribution

It pioneers the OWSOL task and develops a contrastive representation co-learning approach using labeled and unlabeled data, enabling localization of novel categories in open-world settings.

Findings

Outperforms all baseline methods significantly

Creates new evaluation benchmarks for OWSOL

Demonstrates effectiveness on ImageNet-1K, iNatLoc500, and OpenImages150

Abstract

While remarkable success has been achieved in weakly-supervised object localization (WSOL), current frameworks are not capable of locating objects of novel categories in open-world settings. To address this issue, we are the first to introduce a new weakly-supervised object localization task called OWSOL (Open-World Weakly-Supervised Object Localization). During training, all labeled data comes from known categories and, both known and novel categories exist in the unlabeled data. To handle such data, we propose a novel paradigm of contrastive representation co-learning using both labeled and unlabeled data to generate a complete G-CAM (Generalized Class Activation Map) for object localization, without the requirement of bounding box annotation. As no class label is available for the unlabelled data, we conduct clustering over the full training set and design a novel multiple semantic centroids-driven contrastive loss for representation learning. We re-organize two widely used datasets, i.e., ImageNet-1K and iNatLoc500, and propose OpenImages150 to serve as evaluation benchmarks for OWSOL. Extensive experiments demonstrate that the proposed method can surpass all baselines by a large margin. We believe that this work can shift the close-set localization towards the open-world setting and serve as a foundation for subsequent works. Code will be released at https://github.com/ryylcc/OWSOL.