Bridging the Simulation-to-Reality Gap in Electron Microscope Calibration via VAE-EM Estimation

TL;DR

This paper introduces a VAE-EM framework for calibrating electron microscopes that effectively bridges the simulation-to-reality gap, enabling faster and more accurate parameter estimation from noisy, high-dimensional images.

Contribution

It presents a novel joint estimation method using VAEs trained on simulated data and an EM approach to improve calibration accuracy and speed in electron microscopy.

Findings

Achieves 2x reduction in estimation error compared to existing methods.

Requires fewer observations for calibration.

Demonstrates robustness and speed in real STEM calibration.

Abstract

Electron microscopy has enabled many scientific breakthroughs across multiple fields. A key challenge is the tuning of microscope parameters based on images to overcome optical aberrations that deteriorate image quality. This calibration problem is challenging due to the high-dimensional and noisy nature of the diagnostic images, and the fact that optimal parameters cannot be identified from a single image. We tackle the calibration problem for Scanning Transmission Electron Microscopes (STEM) by employing variational autoencoders (VAEs), trained on simulated data, to learn low-dimensional representations of images, whereas most existing methods extract only scalar values. We then simultaneously estimate the model that maps calibration parameters to encoded representations and the optimal calibration parameters using an expectation maximization (EM) approach. This joint estimation explicitly addresses the simulation-to-reality gap inherent in data-driven methods that train on simulated data from a digital twin. We leverage the known symmetry property of the optical system to establish global identifiability of the joint estimation problem, ensuring that a unique optimum exists. We demonstrate that our approach is substantially faster and more consistent than existing methods on a real STEM, achieving a 2x reduction in estimation error while requiring fewer observations. This represents a notable advance in automated STEM calibration and demonstrates the potential of VAEs for information compression in images. Beyond microscopy, the VAE-EM framework applies to inverse problems where simulated training data introduces a reality gap and where non-injective mappings would otherwise prevent unique solutions.