Semantic-Promoted Debiasing and Background Disambiguation for Zero-Shot Instance Segmentation

TL;DR

This paper introduces D$^2$Zero, a novel approach for zero-shot instance segmentation that leverages semantic relationships and background disambiguation to improve detection of unseen objects, significantly outperforming previous methods.

Contribution

The paper proposes a new method combining semantic-promoted debiasing and background disambiguation to enhance zero-shot instance segmentation performance.

Findings

Achieves 16.86% improvement on COCO dataset

Effectively distinguishes novel objects from background

Utilizes semantic relationships for better unseen category detection

Abstract

Zero-shot instance segmentation aims to detect and precisely segment objects of unseen categories without any training samples. Since the model is trained on seen categories, there is a strong bias that the model tends to classify all the objects into seen categories. Besides, there is a natural confusion between background and novel objects that have never shown up in training. These two challenges make novel objects hard to be raised in the final instance segmentation results. It is desired to rescue novel objects from background and dominated seen categories. To this end, we propose D$^2$Zero with Semantic-Promoted Debiasing and Background Disambiguation to enhance the performance of Zero-shot instance segmentation. Semantic-promoted debiasing utilizes inter-class semantic relationships to involve unseen categories in visual feature training and learns an input-conditional classifier to conduct dynamical classification based on the input image. Background disambiguation produces image-adaptive background representation to avoid mistaking novel objects for background. Extensive experiments show that we significantly outperform previous state-of-the-art methods by a large margin, e.g., 16.86% improvement on COCO. Project page: https://henghuiding.github.io/D2Zero/