Open-Set Semi-Supervised Object Detection

TL;DR

This paper introduces a framework for open-set semi-supervised object detection that effectively handles out-of-distribution objects, improving detection performance on large-scale benchmarks by integrating offline OOD detection with SSOD methods.

Contribution

It proposes a novel OSSOD framework combining offline OOD detection with SSOD, addressing semantic expansion issues caused by OOD objects in unlabeled data.

Findings

Offline OOD detector based on vision transformer outperforms online detectors.

Framework improves detection accuracy on COCO-OpenImages benchmarks.

Effective across various OSSOD conditions and dataset configurations.

Abstract

Recent developments for Semi-Supervised Object Detection (SSOD) have shown the promise of leveraging unlabeled data to improve an object detector. However, thus far these methods have assumed that the unlabeled data does not contain out-of-distribution (OOD) classes, which is unrealistic with larger-scale unlabeled datasets. In this paper, we consider a more practical yet challenging problem, Open-Set Semi-Supervised Object Detection (OSSOD). We first find the existing SSOD method obtains a lower performance gain in open-set conditions, and this is caused by the semantic expansion, where the distracting OOD objects are mispredicted as in-distribution pseudo-labels for the semi-supervised training. To address this problem, we consider online and offline OOD detection modules, which are integrated with SSOD methods. With the extensive studies, we found that leveraging an offline OOD detector based on a self-supervised vision transformer performs favorably against online OOD detectors due to its robustness to the interference of pseudo-labeling. In the experiment, our proposed framework effectively addresses the semantic expansion issue and shows consistent improvements on many OSSOD benchmarks, including large-scale COCO-OpenImages. We also verify the effectiveness of our framework under different OSSOD conditions, including varying numbers of in-distribution classes, different degrees of supervision, and different combinations of unlabeled sets.