DOD-SA: Infrared-Visible Decoupled Object Detection with Single-Modality Annotations

TL;DR

This paper introduces DOD-SA, a novel infrared-visible object detection framework that reduces annotation costs by using single-modality annotations and cross-modality knowledge transfer, achieving superior results on the DroneVehicle dataset.

Contribution

The paper proposes a decoupled detection framework with a collaborative teacher-student network and a progressive training strategy to enable effective cross-modality detection with minimal annotations.

Findings

Outperforms state-of-the-art methods on DroneVehicle dataset

Effectively transfers knowledge between modalities with pseudo-labels

Reduces annotation costs while maintaining detection accuracy

Abstract

Infrared-visible object detection has shown great potential in real-world applications, enabling robust all-day perception by leveraging the complementary information of infrared and visible images. However, existing methods typically require dual-modality annotations to output detection results for both modalities during prediction, which incurs high annotation costs. To address this challenge, we propose a novel infrared-visible Decoupled Object Detection framework with Single-modality Annotations, called DOD-SA. The architecture of DOD-SA is built upon a Single- and Dual-Modality Collaborative Teacher-Student Network (CoSD-TSNet), which consists of a single-modality branch (SM-Branch) and a dual-modality decoupled branch (DMD-Branch). The teacher model generates pseudo-labels for the unlabeled modality, simultaneously supporting the training of the student model. The collaborative design enables cross-modality knowledge transfer from the labeled modality to the unlabeled modality, and facilitates effective SM-to-DMD branch supervision. To further improve the decoupling ability of the model and the pseudo-label quality, we introduce a Progressive and Self-Tuning Training Strategy (PaST) that trains the model in three stages: (1) pretraining SM-Branch, (2) guiding the learning of DMD-Branch by SM-Branch, and (3) refining DMD-Branch. In addition, we design a Pseudo Label Assigner (PLA) to align and pair labels across modalities, explicitly addressing modality misalignment during training. Extensive experiments on the DroneVehicle dataset demonstrate that our method outperforms state-of-the-art (SOTA).