Synthetic Industrial Object Detection: GenAI vs. Feature-Based Methods

TL;DR

This paper benchmarks various synthetic data generation and domain adaptation techniques for industrial object detection, finding that simple feature-based methods often outperform complex GenAI approaches in accuracy and efficiency.

Contribution

It provides a comprehensive comparison of feature-based, GenAI, and classical rendering methods for synthetic data creation in industrial contexts, highlighting the effectiveness of simple feature filtering.

Findings

Perceptual hashing achieves up to 98% mAP50 on industrial datasets.

GenAI methods require more time without improving accuracy.

Simple feature-based methods outperform complex GenAI approaches in efficiency and accuracy.

Abstract

Reducing the burden of data generation and annotation remains a major challenge for the cost-effective deployment of machine learning in industrial and robotics settings. While synthetic rendering is a promising solution, bridging the sim-to-real gap often requires expert intervention. In this work, we benchmark a range of domain randomization (DR) and domain adaptation (DA) techniques, including feature-based methods, generative AI (GenAI), and classical rendering approaches, for creating contextualized synthetic data without manual annotation. Our evaluation focuses on the effectiveness and efficiency of low-level and high-level feature alignment, as well as a controlled diffusion-based DA method guided by prompts generated from real-world contexts. We validate our methods on two datasets: a proprietary industrial dataset (automotive and logistics) and a public robotics dataset. Results show that if render-based data with enough variability is available as seed, simpler feature-based methods, such as brightness-based and perceptual hashing filtering, outperform more complex GenAI-based approaches in both accuracy and resource efficiency. Perceptual hashing consistently achieves the highest performance, with mAP50 scores of 98% and 67% on the industrial and robotics datasets, respectively. Additionally, GenAI methods present significant time overhead for data generation at no apparent improvement of sim-to-real mAP values compared to simpler methods. Our findings offer actionable insights for efficiently bridging the sim-to-real gap, enabling high real-world performance from models trained exclusively on synthetic data.