Exploring Few-Shot Defect Segmentation in General Industrial Scenarios with Metric Learning and Vision Foundation Models

TL;DR

This paper explores few-shot defect segmentation in diverse industrial scenarios using metric learning and vision foundation models, introducing a new dataset and benchmark, and evaluating various methods including SAM2.

Contribution

It introduces a new real-world dataset and comprehensive benchmark for few-shot defect segmentation across diverse industrial scenarios, and systematically evaluates the effectiveness of VFMs and proposes a novel feature matching method.

Findings

VFMs outperform meta-learning methods in FDS tasks

SAM2 is highly effective in video mode for defect segmentation

Proposed feature matching method improves segmentation efficiency

Abstract

Industrial defect segmentation is critical for manufacturing quality control. Due to the scarcity of training defect samples, few-shot semantic segmentation (FSS) holds significant value in this field. However, existing studies mostly apply FSS to tackle defects on simple textures, without considering more diverse scenarios. This paper aims to address this gap by exploring FSS in broader industrial products with various defect types. To this end, we contribute a new real-world dataset and reorganize some existing datasets to build a more comprehensive few-shot defect segmentation (FDS) benchmark. On this benchmark, we thoroughly investigate metric learning-based FSS methods, including those based on meta-learning and those based on Vision Foundation Models (VFMs). We observe that existing meta-learning-based methods are generally not well-suited for this task, while VFMs hold great potential. We further systematically study the applicability of various VFMs in this task, involving two paradigms: feature matching and the use of Segment Anything (SAM) models. We propose a novel efficient FDS method based on feature matching. Meanwhile, we find that SAM2 is particularly effective for addressing FDS through its video track mode. The contributed dataset and code will be available at: https://github.com/liutongkun/GFDS.