Exploring Generative Adversarial Networks for Image-to-Image Translation in STEM Simulation

TL;DR

This paper investigates the use of GANs for converting fast, approximate STEM image simulations into high-accuracy multislice images, aiming to balance speed and precision in microscopy image analysis.

Contribution

It introduces a deep learning approach using GANs to improve the accuracy of convolution-based STEM image simulations, matching previous regression methods.

Findings

GANs outperform other deep learning models in accuracy

GAN-based method achieves similar results to regression models

Code and data are publicly available for reproducibility

Abstract

The use of accurate scanning transmission electron microscopy (STEM) image simulation methods require large computation times that can make their use infeasible for the simulation of many images. Other simulation methods based on linear imaging models, such as the convolution method, are much faster but are too inaccurate to be used in application. In this paper, we explore deep learning models that attempt to translate a STEM image produced by the convolution method to a prediction of the high accuracy multislice image. We then compare our results to those of regression methods. We find that using the deep learning model Generative Adversarial Network (GAN) provides us with the best results and performs at a similar accuracy level to previous regression models on the same dataset. Codes and data for this project can be found in this GitHub repository, https://github.com/uw-cmg/GAN-STEM-Conv2MultiSlice.