# UniMR: A Plug‐and‐Play Framework of Automated Molecular Recognition for Scanning Tunneling Microscopy

## TL;DR

UniMR is a new tool that automatically recognizes molecules in STM images without needing training, making it useful for surface chemistry research.

## Contribution

UniMR introduces a training-free framework for molecular recognition in STM images using adaptive feature selection and CLIP embeddings.

## Key findings

- UniMR achieves an average F1-score >0.9 across five molecular systems.
- The framework is robust to low-resolution images and diverse molecular systems.
- UniMR outperforms previous methods in molecular recognition accuracy.

## Abstract

Scanning Tunneling Microscopy (STM) has become an indispensable tool for molecular surface chemistry, where recognizing molecules in STM images is a fundamental task. Human annotation requires a great deal of domain knowledge and is laborious, while current automated methods usually require training and focus on particular molecular systems. In this work, a training‐free universal molecular recognizer (UniMR) is developed that is widely applicable to various molecular systems and is tolerant of low‐resolution images. UniMR integrates off‐the‐shelf computer vision tools with a feature adaptive selection module. First, input images are normalized to ensure consistent molecular brightness and orientation. To address the varying complexity of molecular differentiation across different systems, an adaptive feature selection module is introduced. Specifically, each molecular image is represented by a pixel matrix for shallow structural features and CLIP embeddings for deep semantic features. A Gaussian Mixture Model (GMM) is followed to dynamically select the optimal representation. Finally, cosine similarity identifies molecules of the same type. UniMR is evaluated on five molecular systems with diverse imaging resolutions. The results demonstrate a significant improvement over previous methods, achieving an average F1‐score >0.9. This framework can serve as a versatile auxiliary tool for STM, advancing microscopic surface chemistry research.

UniMR, a training‐free framework for automated molecular recognition in STM images. By integrating adaptive feature selection with CLIP embeddings and Gaussian Mixture Modeling, UniMR achieves robust performance across diverse molecular systems and low‐resolution conditions. Extensive validation demonstrates superior F1‐scores (0.93) over existing methods, highlighting its potential as a versatile tool for advancing surface science research.