Dynamic Spectral fluorescence microscopy via Event-based & CMOS image-sensor fusion

TL;DR

This paper introduces a novel ultra-fast fluorescence microscopy system combining event-based and CMOS sensors with spectral encoding, enabling high-speed, spectral-distinct imaging of dynamic biological processes.

Contribution

The work presents a new microscopy approach integrating event-based sensors with spectral encoding and deep learning for ultra-fast, spectrally resolved fluorescence imaging.

Findings

Achieves ~10 microsecond temporal resolution.

Successfully resolves fluorescence spectra separated by only 7 nm.

Demonstrates imaging of fluorescent bead flow in capillaries.

Abstract

We present a widefield fluorescence microscope that integrates an event-based image sensor (EBIS) with a CMOS image sensor (CIS) for ultra-fast microscopy with spectral distinction capabilities. The EBIS achieves temporal resolution of $\sim10\thinspace\mu$s ($\sim\thinspace$50,000 frames/s), while the CIS provides diffraction-limited spatial resolution. A diffractive optical element encodes spectral information into a diffractogram, which is recorded by the CIS. The diffractogram is processed using a deep neural network to resolve the fluorescence of two beads, whose emission peaks are separated by only 7 nm and exhibit an 88\% spectral overlap. We validate our microscope by imaging the capillary flow of fluorescent beads, demonstrating a significant advancement in ultra-fast spectral microscopy. This technique holds broad potential for elucidating foundational dynamic biological processes.