Semi-DETR: Semi-Supervised Object Detection with Detection Transformers

TL;DR

Semi-DETR introduces a transformer-based semi-supervised object detection framework that improves training efficiency and pseudo label quality through hybrid matching, cross-view query consistency, and cost-based pseudo label mining.

Contribution

It presents the first end-to-end semi-supervised detector based on transformers, with novel strategies for matching, consistency learning, and pseudo label mining.

Findings

Outperforms state-of-the-art methods on COCO and Pascal VOC datasets.

Effective in improving pseudo label quality and training efficiency.

Achieves significant performance gains across various SSOD settings.

Abstract

We analyze the DETR-based framework on semi-supervised object detection (SSOD) and observe that (1) the one-to-one assignment strategy generates incorrect matching when the pseudo ground-truth bounding box is inaccurate, leading to training inefficiency; (2) DETR-based detectors lack deterministic correspondence between the input query and its prediction output, which hinders the applicability of the consistency-based regularization widely used in current SSOD methods. We present Semi-DETR, the first transformer-based end-to-end semi-supervised object detector, to tackle these problems. Specifically, we propose a Stage-wise Hybrid Matching strategy that combines the one-to-many assignment and one-to-one assignment strategies to improve the training efficiency of the first stage and thus provide high-quality pseudo labels for the training of the second stage. Besides, we introduce a Crossview Query Consistency method to learn the semantic feature invariance of object queries from different views while avoiding the need to find deterministic query correspondence. Furthermore, we propose a Cost-based Pseudo Label Mining module to dynamically mine more pseudo boxes based on the matching cost of pseudo ground truth bounding boxes for consistency training. Extensive experiments on all SSOD settings of both COCO and Pascal VOC benchmark datasets show that our Semi-DETR method outperforms all state-of-the-art methods by clear margins. The PaddlePaddle version code1 is at https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/semi_det/semi_detr.