Synth4Seg -- Learning Defect Data Synthesis for Defect Segmentation using Bi-level Optimization

TL;DR

This paper introduces a bi-level optimization framework for synthetic defect data generation that enhances defect segmentation performance by learning optimal defect placement and data source importance, especially under limited data conditions.

Contribution

It proposes a novel bi-level optimization approach for synthetic defect data generation, improving defect segmentation accuracy by learning placement and importance weights.

Findings

Improves segmentation performance by up to 18.3% with optimized defect placement.

Achieves a 2.6% gain by learning importance weights for data sources.

Demonstrates effectiveness on benchmark datasets under limited data settings.

Abstract

Defect segmentation is crucial for quality control in advanced manufacturing, yet data scarcity poses challenges for state-of-the-art supervised deep learning. Synthetic defect data generation is a popular approach for mitigating data challenges. However, many current methods simply generate defects following a fixed set of rules, which may not directly relate to downstream task performance. This can lead to suboptimal performance and may even hinder the downstream task. To solve this problem, we leverage a novel bi-level optimization-based synthetic defect data generation framework. We use an online synthetic defect generation module grounded in the commonly-used Cut\&Paste framework, and adopt an efficient gradient-based optimization algorithm to solve the bi-level optimization problem. We achieve simultaneous training of the defect segmentation network, and learn various parameters of the data synthesis module by maximizing the validation performance of the trained defect segmentation network. Our experimental results on benchmark datasets under limited data settings show that the proposed bi-level optimization method can be used for learning the most effective locations for pasting synthetic defects thereby improving the segmentation performance by up to 18.3\% when compared to pasting defects at random locations. We also demonstrate up to 2.6\% performance gain by learning the importance weights for different augmentation-specific defect data sources when compared to giving equal importance to all the data sources.