Revealing Microscopic Objects in Fluorescence Live Imaging by Video-to-video Translation Based on A Spatial-temporal Generative Adversarial Network

TL;DR

This paper introduces a novel spatial-temporal GAN framework called STGAN that enhances fluorescence microscopy by translating videos to visualize multiple microscopic objects simultaneously, overcoming spectral label limitations.

Contribution

The paper presents a new video-to-video translation method using STGAN to reveal multiple subcellular structures in fluorescence microscopy beyond traditional spectral constraints.

Findings

STGAN effectively translates microscopy videos to visualize multiple objects.

The method mitigates spectral conflicts in fluorescent imaging.

Experimental results demonstrate improved simultaneous visualization of microscopic objects.

Abstract

In spite of being a valuable tool to simultaneously visualize multiple types of subcellular structures using spectrally distinct fluorescent labels, a standard fluoresce microscope is only able to identify a few microscopic objects; such a limit is largely imposed by the number of fluorescent labels available to the sample. In order to simultaneously visualize more objects, in this paper, we propose to use video-to-video translation that mimics the development process of microscopic objects. In essence, we use a microscopy video-to-video translation framework namely Spatial-temporal Generative Adversarial Network (STGAN) to reveal the spatial and temporal relationships between the microscopic objects, after which a microscopy video of one object can be translated to another object in a different domain. The experimental results confirm that the proposed STGAN is effective in microscopy video-to-video translation that mitigates the spectral conflicts caused by the limited fluorescent labels, allowing multiple microscopic objects be simultaneously visualized.