Building Workflows for Interactive Human in the Loop Automated Experiment (hAE) in STEM-EELS

TL;DR

This paper develops a human-in-the-loop framework for automated STEM-EELS experiments, optimizing the discovery of nanoscale structures by monitoring and intervening in deep kernel learning workflows.

Contribution

It introduces a systematic approach to controlling hyperparameters in DKL workflows for STEM-EELS, enabling real-time monitoring and intervention in automated experiments.

Findings

Experiment paths can become trapped in local minima under certain parameters.

Monitoring in system and feature space improves experiment control.

Intervention strategies enhance discovery efficiency.

Abstract

Exploring the structural, chemical, and physical properties of matter on the nano- and atomic scales has become possible with the recent advances in aberration-corrected electron energy-loss spectroscopy (EELS) in scanning transmission electron microscopy (STEM). However, the current paradigm of STEM-EELS relies on the classical rectangular grid sampling, in which all surface regions are assumed to be of equal a priori interest. This is typically not the case for real-world scenarios, where phenomena of interest are concentrated in a small number of spatial locations. One of foundational problems is the discovery of nanometer- or atomic scale structures having specific signatures in EELS spectra. Here we systematically explore the hyperparameters controlling deep kernel learning (DKL) discovery workflows for STEM-EELS and identify the role of the local structural descriptors and acquisition functions on the experiment progression. In agreement with actual experiment, we observe that for certain parameter combinations the experiment path can be trapped in the local minima. We demonstrate the approaches for monitoring automated experiment in the real and feature space of the system and monitor knowledge acquisition of the DKL model. Based on these, we construct intervention strategies, thus defining human-in the loop automated experiment (hAE). This approach can be further extended to other techniques including 4D STEM and other forms of spectroscopic imaging.