Regularized Gradient Statistics Improve Generative Deep Learning Models of Super Resolution Microscopy

TL;DR

This paper introduces a regularization technique that improves deep learning models for generating super-resolution microscopy images by adjusting gradient statistics.

Contribution

The novelty is a regularization method that enhances image quality by aligning gradient and Laplacian statistics with natural scenes.

Findings

Regularizing gradient statistics improves the clarity and small-scale structure of generated super-resolution images.

The proposed method works with any supervised model and is effective for filamentous structures in the BioSR dataset.

Abstract

It is shown that regularizing the signal gradient statistics during training of deep‐learning models of super‐resolution fluorescence microscopy improves the generated images. Specifically, regularizing the images in the training data set is proposed to have gradient and Laplacian statistics closer to those expected for natural‐scene images. The BioSR data set of matched pairs of diffraction‐limited and super‐resolution images is used to evaluate the proposed regularization in a state‐of‐the‐art generative deep‐learning model of super‐resolution microscopy, the Conditional Variational Diffusion Model (CVDM). Since the proposed regularization is applied as a preprocessing step to the training data, it can be used in conjunction with any supervised machine‐learning model. However, its utility is limited to images for which the prior is appropriate, which in the BioSR data set are the images of filamentous structures. The quality and generalization power of CVDM trained with and without the proposed regularization are compared, showing that the new prior yields images with clearer visual detail and better small‐scale structure.

A generative AI model for super‐resolution microscopy images is presented. Super‐resolution microscopy provides high spatial detail at the expense of lower time resolution. Using it for live samples requires computational image reconstruction. It is unclear what good priors and metrics for AI‐generated super‐resolution images are. Here, a perceptive metric accounting for the human visual system provides clues.

Figures