Zero-shot Shape Classification of Nanoparticles in SEM Images using Vision Foundation Models

TL;DR

This paper presents a zero-shot shape classification method for nanoparticles in SEM images using foundation models, eliminating the need for extensive labeled data and training, and demonstrating high accuracy and robustness.

Contribution

The study introduces a novel zero-shot classification pipeline combining SAM and DINOv2 models for nanoparticle shape analysis without extensive fine-tuning.

Findings

Outperforms fine-tuned YOLOv11 and ChatGPT baselines

Robust to small datasets and morphological variations

Effective across diverse nanoparticle datasets

Abstract

Accurate and efficient characterization of nanoparticle morphology in Scanning Electron Microscopy (SEM) images is critical for ensuring product quality in nanomaterial synthesis and accelerating development. However, conventional deep learning methods for shape classification require extensive labeled datasets and computationally demanding training, limiting their accessibility to the typical nanoparticle practitioner in research and industrial settings. In this study, we introduce a zero-shot classification pipeline that leverages two vision foundation models: the Segment Anything Model (SAM) for object segmentation and DINOv2 for feature embedding. By combining these models with a lightweight classifier, we achieve high-precision shape classification across three morphologically diverse nanoparticle datasets - without the need for extensive parameter fine-tuning. Our methodology outperforms a fine-tuned YOLOv11 and ChatGPT o4-mini-high baselines, demonstrating robustness to small datasets, subtle morphological variations, and domain shifts from natural to scientific imaging. Quantitative clustering metrics on PCA plots of the DINOv2 features are discussed as a means of assessing the progress of the chemical synthesis. This work highlights the potential of foundation models to advance automated microscopy image analysis, offering an alternative to traditional deep learning pipelines in nanoparticle research which is both more efficient and more accessible to the user.