CLIP-Guided Source-Free Object Detection in Aerial Images

TL;DR

This paper introduces a source-free object detection method for aerial images that uses CLIP-guided pseudo-label refinement within a self-training framework, improving detection accuracy across diverse domains.

Contribution

It proposes a novel CLIP-guided aggregation technique to enhance pseudo-label quality in source-free domain adaptation for aerial object detection.

Findings

Outperforms existing methods on new DIOR-based datasets

Effective pseudo-label refinement using CLIP guidance

Significant performance improvements demonstrated in experiments

Abstract

Domain adaptation is crucial in aerial imagery, as the visual representation of these images can significantly vary based on factors such as geographic location, time, and weather conditions. Additionally, high-resolution aerial images often require substantial storage space and may not be readily accessible to the public. To address these challenges, we propose a novel Source-Free Object Detection (SFOD) method. Specifically, our approach begins with a self-training framework, which significantly enhances the performance of baseline methods. To alleviate the noisy labels in self-training, we utilize Contrastive Language-Image Pre-training (CLIP) to guide the generation of pseudo-labels, termed CLIP-guided Aggregation (CGA). By leveraging CLIP's zero-shot classification capability, we aggregate its scores with the original predicted bounding boxes, enabling us to obtain refined scores for the pseudo-labels. To validate the effectiveness of our method, we constructed two new datasets from different domains based on the DIOR dataset, named DIOR-C and DIOR-Cloudy. Experimental results demonstrate that our method outperforms other comparative algorithms. The code is available at https://github.com/Lans1ng/SFOD-RS.