Beyond Boundaries: Leveraging Vision Foundation Models for Source-Free Object Detection

TL;DR

This paper introduces a novel source-free object detection framework that leverages vision foundation models to improve domain transferability and label quality, achieving state-of-the-art results.

Contribution

The paper proposes a new SFOD method utilizing VFMs with three modules for feature alignment and pseudo-label fusion, enhancing transferability and discriminability.

Findings

Achieves state-of-the-art SFOD performance on six benchmarks.

Effectively improves feature transferability through VFM-based modules.

Enhances pseudo-label reliability with entropy-aware fusion strategy.

Abstract

Source-Free Object Detection (SFOD) aims to adapt a source-pretrained object detector to a target domain without access to source data. However, existing SFOD methods predominantly rely on internal knowledge from the source model, which limits their capacity to generalize across domains and often results in biased pseudo-labels, thereby hindering both transferability and discriminability. In contrast, Vision Foundation Models (VFMs), pretrained on massive and diverse data, exhibit strong perception capabilities and broad generalization, yet their potential remains largely untapped in the SFOD setting. In this paper, we propose a novel SFOD framework that leverages VFMs as external knowledge sources to jointly enhance feature alignment and label quality. Specifically, we design three VFM-based modules: (1) Patch-weighted Global Feature Alignment (PGFA) distills global features from VFMs using patch-similarity-based weighting to enhance global feature transferability; (2) Prototype-based Instance Feature Alignment (PIFA) performs instance-level contrastive learning guided by momentum-updated VFM prototypes; and (3) Dual-source Enhanced Pseudo-label Fusion (DEPF) fuses predictions from detection VFMs and teacher models via an entropy-aware strategy to yield more reliable supervision. Extensive experiments on six benchmarks demonstrate that our method achieves state-of-the-art SFOD performance, validating the effectiveness of integrating VFMs to simultaneously improve transferability and discriminability.