Benchmark tests of atom segmentation deep learning models with a consistent dataset

TL;DR

This paper introduces a benchmark dataset of STEM images and evaluates recent neural network models for atom localization, revealing their strengths and limitations across different image qualities and structures.

Contribution

It provides a standardized benchmark dataset and comparative analysis of neural network models for atom segmentation in STEM images.

Findings

High performance of models on varied quality images

Performance drops on images outside training data

Availability of dataset and models for community use

Abstract

The information content of atomic resolution scanning transmission electron microscopy (STEM) images can often be reduced to a handful of parameters describing each atomic column, chief amongst which is the column position. Neural networks (NNs) are a high performance, computationally efficient method to automatically locate atomic columns in images, which has led to a profusion of NN models and associated training datasets. We have developed a benchmark dataset of simulated and experimental STEM images and used it to evaluate the performance of two sets of recent NN models for atom location in STEM images. Both models exhibit high performance for images of varying quality from several different crystal lattices. However, there are important differences in performance as a function of image quality, and both models perform poorly for images outside the training data, such as interfaces with large difference in background intensity. Both the benchmark dataset and the models are available using the Foundry service for dissemination, discovery, and reuse of machine learning models.