A Simple Background Augmentation Method for Object Detection with Diffusion Model

TL;DR

This paper introduces a simple background augmentation method using diffusion models to generate diverse training data, significantly improving object detection performance without extra annotations.

Contribution

It proposes a novel data augmentation technique leveraging text-to-image synthesis for enhancing dataset diversity in object detection tasks.

Findings

Background augmentation improves model robustness and generalization.

The method yields notable performance gains on COCO and other benchmarks.

Adjusting prompts and masks ensures generated content aligns with annotations.

Abstract

In computer vision, it is well-known that a lack of data diversity will impair model performance. In this study, we address the challenges of enhancing the dataset diversity problem in order to benefit various downstream tasks such as object detection and instance segmentation. We propose a simple yet effective data augmentation approach by leveraging advancements in generative models, specifically text-to-image synthesis technologies like Stable Diffusion. Our method focuses on generating variations of labeled real images, utilizing generative object and background augmentation via inpainting to augment existing training data without the need for additional annotations. We find that background augmentation, in particular, significantly improves the models' robustness and generalization capabilities. We also investigate how to adjust the prompt and mask to ensure the generated content comply with the existing annotations. The efficacy of our augmentation techniques is validated through comprehensive evaluations of the COCO dataset and several other key object detection benchmarks, demonstrating notable enhancements in model performance across diverse scenarios. This approach offers a promising solution to the challenges of dataset enhancement, contributing to the development of more accurate and robust computer vision models.