Deep Generative SToRM model for dynamic imaging

TL;DR

This paper presents a deep generative SToRM model that reconstructs dynamic images from highly undersampled data by learning a smooth manifold representation using a CNN, reducing memory usage and enhancing regularization.

Contribution

It introduces a novel deep generative framework for dynamic imaging that jointly estimates CNN parameters and latent vectors directly from undersampled data, avoiding extensive training data.

Findings

Significant memory reduction compared to previous SToRM models

Effective spatial regularization via CNN improves image quality

Proposed progressive algorithms reduce computational complexity

Abstract

We introduce a novel generative smoothness regularization on manifolds (SToRM) model for the recovery of dynamic image data from highly undersampled measurements. The proposed generative framework represents the image time series as a smooth non-linear function of low-dimensional latent vectors that capture the cardiac and respiratory phases. The non-linear function is represented using a deep convolutional neural network (CNN). Unlike the popular CNN approaches that require extensive fully-sampled training data that is not available in this setting, the parameters of the CNN generator as well as the latent vectors are jointly estimated from the undersampled measurements using stochastic gradient descent. We penalize the norm of the gradient of the generator to encourage the learning of a smooth surface/manifold, while temporal gradients of the latent vectors are penalized to encourage the time series to be smooth. The main benefits of the proposed scheme are (a) the quite significant reduction in memory demand compared to the analysis based SToRM model, and (b) the spatial regularization brought in by the CNN model. We also introduce efficient progressive approaches to minimize the computational complexity of the algorithm.