DeCo-DETR: Decoupled Cognition DETR for efficient Open-Vocabulary Object Detection

TL;DR

DeCo-DETR introduces a vision-centric, decoupled framework for open-vocabulary object detection that improves inference efficiency and maintains competitive zero-shot detection performance.

Contribution

It proposes a decoupled training strategy and semantic prototype space construction that eliminate reliance on online text encoding, enhancing scalability and efficiency.

Findings

Achieves competitive zero-shot detection performance.

Significantly improves inference efficiency.

Validates effectiveness on standard OVOD benchmarks.

Abstract

Open-vocabulary object detection (OVOD) enables models to recognize objects beyond predefined categories, but existing approaches remain limited in practical deployment. On the one hand, multimodal designs often incur substantial computational overhead due to their reliance on text encoders at inference time. On the other hand, tightly coupled training objectives introduce a trade-off between closed-set detection accuracy and open-world generalization. Thus, we propose Decoupled Cognition DETR (DeCo-DETR), a vision-centric framework that addresses these challenges through a unified decoupling paradigm. Instead of depending on online text encoding, DeCo-DETR constructs a hierarchical semantic prototype space from region-level descriptions generated by pre-trained LVLMs and aligned via CLIP, enabling efficient and reusable semantic representation. Building upon this representation, the framework further disentangles semantic reasoning from localization through a decoupled training strategy, which separates alignment and detection into parallel optimization streams. Extensive experiments on standard OVOD benchmarks demonstrate that DeCo-DETR achieves competitive zero-shot detection performance while significantly improving inference efficiency. These results highlight the effectiveness of decoupling semantic cognition from detection, offering a practical direction for scalable OVOD systems.