UniT: Unified Knowledge Transfer for Any-shot Object Detection and Segmentation

TL;DR

UniT introduces a unified semi-supervised framework that effectively transfers knowledge across different levels of supervision for object detection and segmentation, enabling robust performance from zero to few-shot scenarios.

Contribution

It proposes an end-to-end trainable model that bridges weakly-supervised and fully-supervised methods, leveraging class similarities for improved novel class detection and segmentation.

Findings

Enhanced performance across zero to few-shot settings

Effective knowledge transfer between base and novel classes

Consistent improvements on MS-COCO and Pascal VOC datasets

Abstract

Methods for object detection and segmentation rely on large scale instance-level annotations for training, which are difficult and time-consuming to collect. Efforts to alleviate this look at varying degrees and quality of supervision. Weakly-supervised approaches draw on image-level labels to build detectors/segmentors, while zero/few-shot methods assume abundant instance-level data for a set of base classes, and none to a few examples for novel classes. This taxonomy has largely siloed algorithmic designs. In this work, we aim to bridge this divide by proposing an intuitive and unified semi-supervised model that is applicable to a range of supervision: from zero to a few instance-level samples per novel class. For base classes, our model learns a mapping from weakly-supervised to fully-supervised detectors/segmentors. By learning and leveraging visual and lingual similarities between the novel and base classes, we transfer those mappings to obtain detectors/segmentors for novel classes; refining them with a few novel class instance-level annotated samples, if available. The overall model is end-to-end trainable and highly flexible. Through extensive experiments on MS-COCO and Pascal VOC benchmark datasets we show improved performance in a variety of settings.