Object-Centric Data Synthesis for Category-level Object Detection

TL;DR

This paper explores how different data synthesis techniques using object-centric data can improve category-level object detection, especially for new classes with limited training data, by evaluating their effectiveness in real-world scenarios.

Contribution

It systematically evaluates four data synthesis methods—image processing, 3D rendering, and diffusion models—for enhancing object detection on novel categories with limited data.

Findings

Synthesized data significantly improves detection accuracy.

Object-centric data enables better generalization to new categories.

Different synthesis methods vary in effectiveness depending on context.

Abstract

Deep learning approaches to object detection have achieved reliable detection of specific object classes in images. However, extending a model's detection capability to new object classes requires large amounts of annotated training data, which is costly and time-consuming to acquire, especially for long-tailed classes with insufficient representation in existing datasets. Here, we introduce the object-centric data setting, when limited data is available in the form of object-centric data (multi-view images or 3D models), and systematically evaluate the performance of four different data synthesis methods to finetune object detection models on novel object categories in this setting. The approaches are based on simple image processing techniques, 3D rendering, and image diffusion models, and use object-centric data to synthesize realistic, cluttered images with varying contextual coherence and complexity. We assess how these methods enable models to achieve category-level generalization in real-world data, and demonstrate significant performance boosts within this data-constrained experimental setting.