VFM$^{4}$SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection

TL;DR

This paper introduces VFM$^{4}$SDG, a novel framework leveraging a frozen vision foundation model to improve single-domain generalized object detection under diverse and unseen environmental conditions.

Contribution

It proposes a dual-prior learning approach that enhances detector stability across domains by integrating a frozen vision foundation model into encoding and decoding stages.

Findings

Outperforms state-of-the-art methods on SDGOD benchmarks

Improves robustness of object-background and inter-instance relations

Enhances semantic recognition and spatial localization stability

Abstract

In real-world scenarios, continual changes in weather, illumination, and imaging conditions cause significant domain shifts, leading detectors trained on a single source domain to degrade severely in unseen environments. Existing single-domain generalized object detection (SDGOD) methods mainly rely on data augmentation or domain-invariant representation learning, but pay limited attention to detector mechanisms, leaving clear limitations under complex domain shifts. Through analytical experiments, we find that performance degradation is dominated by increasing missed detections, which fundamentally arises from reduced cross-domain stability of the detector: object-background and inter-instance relations become less stable in the encoding stage, while semantic-spatial alignment of query representations also becomes harder to maintain in the decoding stage. To this end, we propose VFM$^{4}$SDG, a dual-prior learning framework for SDGOD, which introduces a frozen vision foundation model (VFM) as a transferable cross-domain stability prior into detector representation learning and query modeling. In the encoding stage, we propose Cross-domain Stable Relational Prior Distillation to enhance the robustness of object-background and inter-instance relational modeling. In the decoding stage, we propose Semantic-Contextual Prior-based Query Enhancement, which injects category-level semantic prototypes and global visual context into queries to improve their semantic recognition and spatial localization stability in unseen domains. Extensive experiments show that the proposed method consistently outperforms existing SOTA methods on standard SDGOD benchmarks and two mainstream DETR-based detectors, demonstrating its effectiveness, robustness, and generality.