Synthetic dataset generation for object-to-model deep learning in industrial applications

TL;DR

This paper presents a synthetic data generation framework using 3D models and photogrammetry to train deep learning models for industrial object detection, achieving high accuracy with minimal manual annotation.

Contribution

The work introduces an end-to-end pipeline for creating synthetic datasets from real-world objects to train effective deep learning models for industrial applications.

Findings

Achieved 95.8% classification accuracy on real test images.

Generated 100k synthetic images from 60 real images per class.

Trained a real-time detector using synthetic, annotated data.

Abstract

The availability of large image data sets has been a crucial factor in the success of deep learning-based classification and detection methods. While data sets for everyday objects are widely available, data for specific industrial use-cases (e.g. identifying packaged products in a warehouse) remains scarce. In such cases, the data sets have to be created from scratch, placing a crucial bottleneck on the deployment of deep learning techniques in industrial applications. We present work carried out in collaboration with a leading UK online supermarket, with the aim of creating a computer vision system capable of detecting and identifying unique supermarket products in a warehouse setting. To this end, we demonstrate a framework for using synthetic data to create an end-to-end deep learning pipeline, beginning with real-world objects and culminating in a trained model. Our method is based on the generation of a synthetic dataset from 3D models obtained by applying photogrammetry techniques to real-world objects. Using 100k synthetic images generated from 60 real images per class, an InceptionV3 convolutional neural network (CNN) was trained, which achieved classification accuracy of 95.8% on a separately acquired test set of real supermarket product images. The image generation process supports automatic pixel annotation. This eliminates the prohibitively expensive manual annotation typically required for detection tasks. Based on this readily available data, a one-stage RetinaNet detector was trained on the synthetic, annotated images to produce a detector that can accurately localize and classify the specimen products in real-time.