Neural Field-Based 3D Surface Reconstruction of Microstructures from Multi-Detector Signals in Scanning Electron Microscopy

TL;DR

NFH-SEM is a neural field-based framework that enables high-fidelity 3D surface reconstruction of microstructures from SEM images, overcoming limitations of existing methods by integrating physics-based modeling and multi-view geometry.

Contribution

The paper introduces NFH-SEM, a novel hybrid neural field approach that incorporates SEM physics for self-calibrated, shadow-robust 3D reconstruction from multi-detector SEM images.

Findings

Reconstructed microstructures with nanometer-scale accuracy.

Successfully recovered layered features, surface textures, and fracture steps across diverse specimens.

Code and dataset are publicly available at https://github.com/zju3dv/NFH-SEM.

Abstract

The 3D characterization of microstructures is crucial for understanding and designing functional materials. However, the scanning electron microscope (SEM), widely used in scientific research, captures only 2D electron intensity distributions. Existing SEM 3D reconstruction methods struggle with textureless regions, shadowing artifacts, and calibration dependencies, whereas advanced learning-based approaches fail to generalize to microscopic SEM domains due to the lack of physical priors and domain-specific data. We introduce NFH-SEM, a neural field-based hybrid framework that reconstructs high-fidelity 3D surfaces from multi-view, multi-detector SEM images. NFH-SEM integrates coarse multi-view geometry with photometric stereo cues from detector signals through a continuous neural field, incorporating a learnable forward model that embeds SEM imaging physics for self-calibrated, shadow-robust reconstruction. NFH-SEM achieves precise recovery across diverse specimens, revealing 478 nm layered features in two-photon lithography samples, 782 nm surface textures on pollen grains, and 1.559 $\mu$m fracture steps on silicon carbide particles, demonstrating its accuracy and broad applicability. Our code and real-world dataset are available at https://github.com/zju3dv/NFH-SEM.