# Deep Learned Optical Multiplexing for Multi-Focal Plane Microscopy

## TL;DR

This paper introduces a deep learning-based method for multi-focal plane microscopy that uses a single LED illumination pattern to multiplex information from multiple focal planes into one image, enabling real-time live imaging.

## Contribution

It proposes jointly optimizing LED illumination patterns with a neural network to achieve fast, multiplexed multi-focal imaging in live microscopy.

## Key findings

- Achieved live imaging of D. japonica planarians at 5 focal planes.
- Reduced imaging time by multiplexing multiple focal planes into one image.
- Demonstrated effective post-processing neural network for refocusing.

## Abstract

To obtain microscope images at multiple focal planes, the distance between the objective and sample can be mechanically adjusted. Images are acquired sequentially at each axial distance. Digital refocusing with a light-emitting diode (LED) array microscope allows elimination of this mechanical movement. In an LED array microscope, the light source of a conventional widefield microscope is replaced with a 2-dimensional LED matrix. A stack of images is acquired from the LED array microscope by sequentially illuminating each LED and capturing an image. Previous work has shown that we can achieve digital refocusing by post-processing this LED image stack. Though mechanical scanning is eliminated, digital refocusing with an LED array microscope has low temporal resolution due to the acquisition of multiple images. In this work, we propose a new paradigm for multi-focal plane microscopy for live imaging, utilizing an LED array microscope and deep learning. In our deep learning approach, we look for a single LED illumination pattern that allows the information from multiple focal planes to be multiplexed into a single image. We jointly optimize this LED illumination pattern with the parameters of a post-processing deep neural network, using a training set of LED image stacks from fixed, not live, Dugesia japonica planarians. Once training is complete, we obtain multiple focal planes by inputting a single multiplexed LED image into the trained post-processing deep neural network. We demonstrate live imaging of a D. japonica planarian at 5 focal planes with our method.