Improving Deep Learning-based Defect Detection on Window Frames with Image Processing Strategies

TL;DR

This paper introduces InspectNet, an improved deep learning pipeline combining image enhancement and augmentation techniques with a tuned Unet model, significantly boosting defect detection accuracy in challenging window frame inspection environments.

Contribution

The study presents a novel defect detection pipeline, InspectNet, integrating optimized image processing and augmentation with a tailored Unet model for enhanced defect segmentation.

Findings

Unet with IPT-enhanced augmentations achieved an average IoU of 0.91.

The proposed pipeline outperformed other algorithms across multiple evaluation metrics.

Data augmentation significantly improved defect detection accuracy.

Abstract

Detecting subtle defects in window frames, including dents and scratches, is vital for upholding product integrity and sustaining a positive brand perception. Conventional machine vision systems often struggle to identify these defects in challenging environments like construction sites. In contrast, modern vision systems leveraging machine and deep learning (DL) are emerging as potent tools, particularly for cosmetic inspections. However, the promise of DL is yet to be fully realized. A few manufacturers have established a clear strategy for AI integration in quality inspection, hindered mainly by issues like scarce clean datasets and environmental changes that compromise model accuracy. Addressing these challenges, our study presents an innovative approach that amplifies defect detection in DL models, even with constrained data resources. The paper proposes a new defect detection pipeline called InspectNet (IPT-enhanced UNET) that includes the best combination of image enhancement and augmentation techniques for pre-processing the dataset and a Unet model tuned for window frame defect detection and segmentation. Experiments were carried out using a Spot Robot doing window frame inspections . 16 variations of the dataset were constructed using different image augmentation settings. Results of the experiments revealed that, on average, across all proposed evaluation measures, Unet outperformed all other algorithms when IPT-enhanced augmentations were applied. In particular, when using the best dataset, the average Intersection over Union (IoU) values achieved were IPT-enhanced Unet, reaching 0.91 of mIoU.