DefFiller: Mask-Conditioned Diffusion for Salient Steel Surface Defect Generation

TL;DR

DefFiller introduces a mask-conditioned diffusion model that generates realistic steel defect images without pixel-level annotations, improving defect detection models' performance in industrial settings.

Contribution

We propose DefFiller, a novel diffusion-based method for defect image generation conditioned on masks, reducing annotation effort and enhancing detection accuracy.

Findings

Generated defect images match mask conditions accurately.

Augmented datasets with DefFiller improve detection model performance.

High-quality samples demonstrate effectiveness of the approach.

Abstract

Current saliency-based defect detection methods show promise in industrial settings, but the unpredictability of defects in steel production environments complicates dataset creation, hampering model performance. Existing data augmentation approaches using generative models often require pixel-level annotations, which are time-consuming and resource-intensive. To address this, we introduce DefFiller, a mask-conditioned defect generation method that leverages a layout-to-image diffusion model. DefFiller generates defect samples paired with mask conditions, eliminating the need for pixel-level annotations and enabling direct use in model training. We also develop an evaluation framework to assess the quality of generated samples and their impact on detection performance. Experimental results on the SD-Saliency-900 dataset demonstrate that DefFiller produces high-quality defect images that accurately match the provided mask conditions, significantly enhancing the performance of saliency-based defect detection models trained on the augmented dataset.