Contrast transfer functions help quantify neural network out-of-distribution generalization in HRTEM

TL;DR

This study investigates how neural networks trained on synthetic HRTEM data generalize to different imaging conditions, using contrast transfer functions to quantify domain shifts and assess model stability.

Contribution

The paper introduces a framework utilizing contrast transfer functions to quantify and analyze OOD generalization in neural networks for HRTEM imaging.

Findings

Neural networks maintain performance stability under moderate domain shifts.

Performance degrades smoothly as imaging conditions diverge from training data.

Contrast transfer functions effectively quantify the extent of domain shifts.

Abstract

Neural networks, while effective for tackling many challenging scientific tasks, are not known to perform well out-of-distribution (OOD), i.e., within domains which differ from their training data. Understanding neural network OOD generalization is paramount to their successful deployment in experimental workflows, especially when ground-truth knowledge about the experiment is hard to establish or experimental conditions significantly vary. With inherent access to ground-truth information and fine-grained control of underlying distributions, simulation-based data curation facilitates precise investigation of OOD generalization behavior. Here, we probe generalization with respect to imaging conditions of neural network segmentation models for high-resolution transmission electron microscopy (HRTEM) imaging of nanoparticles, training and measuring the OOD generalization of over 12,000 neural networks using synthetic data generated via random structure sampling and multislice simulation. Using the HRTEM contrast transfer function, we further develop a framework to compare information content of HRTEM datasets and quantify OOD domain shifts. We demonstrate that neural network segmentation models enjoy significant performance stability, but will smoothly and predictably worsen as imaging conditions shift from the training distribution. Lastly, we consider limitations of our approach in explaining other OOD shifts, such as of the atomic structures, and discuss complementary techniques for understanding generalization in such settings.