Recurrent neural network-based volumetric fluorescence microscopy

TL;DR

This paper introduces Recurrent-MZ, a deep learning framework using recurrent neural networks to digitally reconstruct 3D volumetric images from sparse 2D fluorescence microscopy data, significantly extending depth-of-field and reducing scanning requirements.

Contribution

It presents the first application of recurrent neural networks for microscopic image reconstruction, enabling flexible, rapid, and high-quality volumetric imaging from limited axial data.

Findings

Increases depth-of-field by approximately 50-fold.

Reduces the number of axial scans needed by 30 times.

Demonstrates robustness to varying imaging conditions.

Abstract

Volumetric imaging of samples using fluorescence microscopy plays an important role in various fields including physical, medical and life sciences. Here we report a deep learning-based volumetric image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope at arbitrary axial positions within the sample volume. Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume over an extended depth-of-field. Using experiments on C. Elegans and nanobead samples, Recurrent-MZ is demonstrated to increase the depth-of-field of a 63x/1.4NA objective lens by approximately 50-fold, also providing a 30-fold reduction in the number of axial scans required to image the same sample volume. We further illustrated the generalization of this recurrent network for 3D imaging by showing its resilience to varying imaging conditions, including e.g., different sequences of input images, covering various axial permutations and unknown axial positioning errors. Recurrent-MZ demonstrates the first application of recurrent neural networks in microscopic image reconstruction and provides a flexible and rapid volumetric imaging framework, overcoming the limitations of current 3D scanning microscopy tools.