Learning to Discover and Detect Objects

TL;DR

This paper introduces RNCDL, a two-stage detection network that discovers and localizes novel object classes using limited supervision, outperforming traditional clustering methods on multiple datasets.

Contribution

The paper presents a novel end-to-end detection framework for discovering and localizing unseen classes without explicit supervision, leveraging a class distribution constraint.

Findings

Significantly outperforms multi-stage clustering pipelines

Effective on COCO, LVIS, and Visual Genome datasets

Learns to detect diverse semantic classes without direct supervision

Abstract

We tackle the problem of novel class discovery and localization (NCDL). In this setting, we assume a source dataset with supervision for only some object classes. Instances of other classes need to be discovered, classified, and localized automatically based on visual similarity without any human supervision. To tackle NCDL, we propose a two-stage object detection network Region-based NCDL (RNCDL) that uses a region proposal network to localize regions of interest (RoIs). We then train our network to learn to classify each RoI, either as one of the known classes, seen in the source dataset, or one of the novel classes, with a long-tail distribution constraint on the class assignments, reflecting the natural frequency of classes in the real world. By training our detection network with this objective in an end-to-end manner, it learns to classify all region proposals for a large variety of classes, including those not part of the labeled object class vocabulary. Our experiments conducted using COCO and LVIS datasets reveal that our method is significantly more effective than multi-stage pipelines that rely on traditional clustering algorithms. Furthermore, we demonstrate the generality of our approach by applying our method to a large-scale Visual Genome dataset, where our network successfully learns to detect various semantic classes without direct supervision.