Open-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning

TL;DR

This paper introduces Openset RCNN, a novel open-set object detection framework that uses classification-free region proposals and contrastive learning to identify known and unknown objects, enhancing robotic perception in unstructured environments.

Contribution

The paper proposes a new open-set object detection method combining CF-RPN and prototype learning network for better unknown object recognition.

Findings

Outperforms existing methods on open-set detection benchmarks.

Enables robots to recognize unknown objects in cluttered scenes.

Provides a new benchmark based on reorganized GraspNet-1billion dataset.

Abstract

Detecting both known and unknown objects is a fundamental skill for robot manipulation in unstructured environments. Open-set object detection (OSOD) is a promising direction to handle the problem consisting of two subtasks: objects and background separation, and open-set object classification. In this paper, we present Openset RCNN to address the challenging OSOD. To disambiguate unknown objects and background in the first subtask, we propose to use classification-free region proposal network (CF-RPN) which estimates the objectness score of each region purely using cues from object's location and shape preventing overfitting to the training categories. To identify unknown objects in the second subtask, we propose to represent them using the complementary region of known categories in a latent space which is accomplished by a prototype learning network (PLN). PLN performs instance-level contrastive learning to encode proposals to a latent space and builds a compact region centering with a prototype for each known category. Further, we note that the detection performance of unknown objects can not be unbiasedly evaluated on the situation that commonly used object detection datasets are not fully annotated. Thus, a new benchmark is introduced by reorganizing GraspNet-1billion, a robotic grasp pose detection dataset with complete annotation. Extensive experiments demonstrate the merits of our method. We finally show that our Openset RCNN can endow the robot with an open-set perception ability to support robotic rearrangement tasks in cluttered environments. More details can be found in https://sites.google.com/view/openset-rcnn/