ODGEN: Domain-specific Object Detection Data Generation with Diffusion Models

TL;DR

ODGEN is a diffusion model-based data generation method that creates high-quality, controllable images with multi-class and occluded objects, significantly improving object detection performance across various domains.

Contribution

The paper introduces ODGEN, a novel diffusion-based approach for domain-specific image synthesis conditioned on bounding boxes, enhancing data quality and controllability for object detection.

Findings

Improves mAP@.50:.95 by up to 25.3% on object detectors.

Outperforms prior controllable generative methods.

Demonstrates robustness in complex scenes and specific domains.

Abstract

Modern diffusion-based image generative models have made significant progress and become promising to enrich training data for the object detection task. However, the generation quality and the controllability for complex scenes containing multi-class objects and dense objects with occlusions remain limited. This paper presents ODGEN, a novel method to generate high-quality images conditioned on bounding boxes, thereby facilitating data synthesis for object detection. Given a domain-specific object detection dataset, we first fine-tune a pre-trained diffusion model on both cropped foreground objects and entire images to fit target distributions. Then we propose to control the diffusion model using synthesized visual prompts with spatial constraints and object-wise textual descriptions. ODGEN exhibits robustness in handling complex scenes and specific domains. Further, we design a dataset synthesis pipeline to evaluate ODGEN on 7 domain-specific benchmarks to demonstrate its effectiveness. Adding training data generated by ODGEN improves up to 25.3% mAP@.50:.95 with object detectors like YOLOv5 and YOLOv7, outperforming prior controllable generative methods. In addition, we design an evaluation protocol based on COCO-2014 to validate ODGEN in general domains and observe an advantage up to 5.6% in mAP@.50:.95 against existing methods.