Synthetic Data Generation for Bridging Sim2Real Gap in a Production Environment

TL;DR

This paper explores synthetic data generation methods tailored for complex production environments, demonstrating that combining basic procedures can significantly improve the simulation-to-reality transfer for training vision models.

Contribution

It introduces and evaluates synthetic data generation procedures specifically designed for parts and assemblies in production settings, highlighting the importance of domain knowledge.

Findings

Up to 15% improvement with combined synthetic data procedures

Evaluation on real production images shows enhanced model generalization

Effective synthetic data generation reduces the simulation-to-reality gap

Abstract

Synthetic data is being used lately for training deep neural networks in computer vision applications such as object detection, object segmentation and 6D object pose estimation. Domain randomization hereby plays an important role in reducing the simulation to reality gap. However, this generalization might not be effective in specialized domains like a production environment involving complex assemblies. Either the individual parts, trained with synthetic images, are integrated in much larger assemblies making them indistinguishable from their counterparts and result in false positives or are partially occluded just enough to give rise to false negatives. Domain knowledge is vital in these cases and if conceived effectively while generating synthetic data, can show a considerable improvement in bridging the simulation to reality gap. This paper focuses on synthetic data generation procedures for parts and assemblies used in a production environment. The basic procedures for synthetic data generation and their various combinations are evaluated and compared on images captured in a production environment, where results show up to 15% improvement using combinations of basic procedures. Reducing the simulation to reality gap in this way can aid to utilize the true potential of robot assisted production using artificial intelligence.