SegSEM: Enabling and Enhancing SAM2 for SEM Contour Extraction

TL;DR

This paper introduces SegSEM, a framework that adapts SAM2 for SEM contour extraction using few-shot learning, combining data-efficient fine-tuning and a hybrid architecture with fallback, demonstrating effectiveness on a small dataset.

Contribution

It presents a novel methodology for adapting foundation models like SAM2 to specialized industrial tasks with limited annotated data.

Findings

SegSEM successfully adapts SAM2 for SEM contour extraction with only 60 images.

The hybrid architecture improves robustness by integrating traditional algorithms as fallback.

Data-efficient fine-tuning of encoders enhances model adaptation in data-scarce scenarios.

Abstract

Extracting high-fidelity 2D contours from Scanning Electron Microscope (SEM) images is critical for calibrating Optical Proximity Correction (OPC) models. While foundation models like Segment Anything 2 (SAM2) are promising, adapting them to specialized domains with scarce annotated data is a major challenge. This paper presents a case study on adapting SAM2 for SEM contour extraction in a few-shot setting. We propose SegSEM, a framework built on two principles: a data-efficient fine-tuning strategy that adapts by selectively training only the model's encoders, and a robust hybrid architecture integrating a traditional algorithm as a confidence-aware fallback. Using a small dataset of 60 production images, our experiments demonstrate this methodology's viability. The primary contribution is a methodology for leveraging foundation models in data-constrained industrial applications.