A robust synthetic data generation framework for machine learning in High-Resolution Transmission Electron Microscopy (HRTEM)

TL;DR

This paper presents Construction Zone, a Python tool for generating diverse synthetic nanoscale structures to train neural networks for HRTEM image analysis, achieving state-of-the-art results without real experimental data.

Contribution

Introduction of Construction Zone, a fast, systematic synthetic data generator for nanomaterials, and an end-to-end workflow for training neural networks on simulated HRTEM data.

Findings

Achieved state-of-the-art segmentation on experimental HRTEM images.

Demonstrated the importance of simulation fidelity and data distribution.

Provided strategies for high-performance machine learning with synthetic data.

Abstract

Machine learning techniques are attractive options for developing highly-accurate automated analysis tools for nanomaterials characterization, including high-resolution transmission electron microscopy (HRTEM). However, successfully implementing such machine learning tools can be difficult due to the challenges in procuring sufficiently large, high-quality training datasets from experiments. In this work, we introduce Construction Zone, a Python package for rapidly generating complex nanoscale atomic structures, and develop an end-to-end workflow for creating large simulated databases for training neural networks. Construction Zone enables fast, systematic sampling of realistic nanomaterial structures, and can be used as a random structure generator for simulated databases, which is important for generating large, diverse synthetic datasets. Using HRTEM imaging as an example, we train a series of neural networks on various subsets of our simulated databases to segment nanoparticles and holistically study the data curation process to understand how various aspects of the curated simulated data -- including simulation fidelity, the distribution of atomic structures, and the distribution of imaging conditions -- affect model performance across several experimental benchmarks. Using our results, we are able to achieve state-of-the-art segmentation performance on experimental HRTEM images of nanoparticles from several experimental benchmarks and, further, we discuss robust strategies for consistently achieving high performance with machine learning in experimental settings using purely synthetic data.