Semantics-Guided Contrastive Network for Zero-Shot Object detection

TL;DR

This paper introduces ContrastZSD, a novel semantics-guided contrastive network for zero-shot object detection that improves the discrimination and generalization to unseen categories by leveraging semantic supervision.

Contribution

The paper proposes a new contrastive learning framework for ZSD that incorporates semantic guidance to better utilize unseen class information and structure visual features.

Findings

Outperforms state-of-the-art on PASCAL VOC and MS COCO benchmarks.

Effectively reduces bias towards seen categories in ZSD.

Enhances visual-semantic alignment for improved detection of unseen objects.

Abstract

Zero-shot object detection (ZSD), the task that extends conventional detection models to detecting objects from unseen categories, has emerged as a new challenge in computer vision. Most existing approaches tackle the ZSD task with a strict mapping-transfer strategy, which may lead to suboptimal ZSD results: 1) the learning process of those models ignores the available unseen class information, and thus can be easily biased towards the seen categories; 2) the original visual feature space is not well-structured and lack of discriminative information. To address these issues, we develop a novel Semantics-Guided Contrastive Network for ZSD, named ContrastZSD, a detection framework that first brings contrastive learning mechanism into the realm of zero-shot detection. Particularly, ContrastZSD incorporates two semantics-guided contrastive learning subnets that contrast between region-category and region-region pairs respectively. The pairwise contrastive tasks take advantage of additional supervision signals derived from both ground truth label and pre-defined class similarity distribution. Under the guidance of those explicit semantic supervision, the model can learn more knowledge about unseen categories to avoid the bias problem to seen concepts, while optimizing the data structure of visual features to be more discriminative for better visual-semantic alignment. Extensive experiments are conducted on two popular benchmarks for ZSD, i.e., PASCAL VOC and MS COCO. Results show that our method outperforms the previous state-of-the-art on both ZSD and generalized ZSD tasks.