High-Quality Proposal Encoding and Cascade Denoising for Imaginary Supervised Object Detection

TL;DR

This paper introduces Cascade HQP-DETR, a novel approach for Imaginary Supervised Object Detection that leverages high-quality data generation, image-specific query encoding, and cascade denoising to improve real-world detection performance from synthetic training data.

Contribution

The paper proposes a comprehensive framework combining high-quality synthetic data, advanced query encoding, and dynamic denoising to enhance ISOD, achieving state-of-the-art results with limited training epochs.

Findings

Achieves 61.04% mAP@0.5 on PASCAL VOC 2007

Outperforms existing baselines in synthetic-to-real transfer

Demonstrates universal applicability across datasets

Abstract

Object detection models demand large-scale annotated datasets, which are costly and labor-intensive to create. This motivated Imaginary Supervised Object Detection (ISOD), where models train on synthetic images and test on real images. However, existing methods face three limitations: (1) synthetic datasets suffer from simplistic prompts, poor image quality, and weak supervision; (2) DETR-based detectors, due to their random query initialization, struggle with slow convergence and overfitting to synthetic patterns, hindering real-world generalization; (3) uniform denoising pressure promotes model overfitting to pseudo-label noise. We propose Cascade HQP-DETR to address these limitations. First, we introduce a high-quality data pipeline using LLaMA-3, Flux, and Grounding DINO to generate the FluxVOC and FluxCOCO datasets, advancing ISOD from weak to full supervision. Second, our High-Quality Proposal guided query encoding initializes object queries with image-specific priors from SAM-generated proposals and RoI-pooled features, accelerating convergence while steering the model to learn transferable features instead of overfitting to synthetic patterns. Third, our cascade denoising algorithm dynamically adjusts training weights through progressively increasing IoU thresholds across decoder layers, guiding the model to learn robust boundaries from reliable visual cues rather than overfitting to noisy labels. Trained for just 12 epochs solely on FluxVOC, Cascade HQP-DETR achieves a SOTA 61.04\% mAP@0.5 on PASCAL VOC 2007, outperforming strong baselines, with its competitive real-data performance confirming the architecture's universal applicability.