Deep Generative Models-Assisted Automated Labeling for Electron Microscopy Images Segmentation

TL;DR

This paper introduces a fully automated, label-free deep learning pipeline using tandem GANs to generate synthetic EM images, improving segmentation accuracy and adaptability across different data domains without manual labeling.

Contribution

The proposed tGAN framework eliminates manual labeling and simulation needs, enabling tailored EM image generation for training and cross-domain application without manual intervention.

Findings

Recognition accuracy exceeds manual labeling by 5%.

The method adapts to various data domains via transfer learning.

It reduces the need for tedious dataset annotations.

Abstract

The rapid advancement of deep learning has facilitated the automated processing of electron microscopy (EM) big data stacks. However, designing a framework that eliminates manual labeling and adapts to domain gaps remains challenging. Current research remains entangled in the dilemma of pursuing complete automation while still requiring simulations or slight manual annotations. Here we demonstrate tandem generative adversarial network (tGAN), a fully label-free and simulation-free pipeline capable of generating EM images for computer vision training. The tGAN can assimilate key features from new data stacks, thus producing a tailored virtual dataset for the training of automated EM analysis tools. Using segmenting nanoparticles for analyzing size distribution of supported catalysts as the demonstration, our findings showcased that the recognition accuracy of tGAN even exceeds the manually-labeling method by 5%. It can also be adaptively deployed to various data domains without further manual manipulation, which is verified by transfer learning from HAADF-STEM to BF-TEM. This generalizability may enable it to extend its application to a broader range of imaging characterizations, liberating microscopists and materials scientists from tedious dataset annotations.