Controllable Image Synthesis of Industrial Data Using Stable Diffusion

TL;DR

This paper introduces a method to adapt pre-trained generative models for industrial defect image synthesis, enabling improved defect detection with limited data by generating labeled defective images tailored to specific industrial topologies.

Contribution

The authors propose a novel approach to fine-tune pre-trained generative models for industrial defect image synthesis, facilitating targeted data augmentation for defect detection tasks.

Findings

Synthetic data improves crack segmentation performance.

Method enhances defect detection with small datasets.

Generative model adapts to industrial defect characteristics.

Abstract

Training supervised deep neural networks that perform defect detection and segmentation requires large-scale fully-annotated datasets, which can be hard or even impossible to obtain in industrial environments. Generative AI offers opportunities to enlarge small industrial datasets artificially, thus enabling the usage of state-of-the-art supervised approaches in the industry. Unfortunately, also good generative models need a lot of data to train, while industrial datasets are often tiny. Here, we propose a new approach for reusing general-purpose pre-trained generative models on industrial data, ultimately allowing the generation of self-labelled defective images. First, we let the model learn the new concept, entailing the novel data distribution. Then, we force it to learn to condition the generative process, producing industrial images that satisfy well-defined topological characteristics and show defects with a given geometry and location. To highlight the advantage of our approach, we use the synthetic dataset to optimise a crack segmentor for a real industrial use case. When the available data is small, we observe considerable performance increase under several metrics, showing the method's potential in production environments.