Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms

TL;DR

This paper develops a theoretical framework for untrained neural network-based snapshot compressive imaging, optimizing mask parameters and connecting data recovery to network parameters, leading to state-of-the-art results.

Contribution

It introduces a theoretical analysis of UNN-based SCI, optimizing measurement masks and proposing SCI-BDVP algorithms that outperform existing methods.

Findings

Achieves state-of-the-art results among UNN methods in video SCI.

Outperforms supervised solutions in noisy measurement scenarios.

Provides a theoretical foundation linking network parameters to data recovery capabilities.

Abstract

Snapshot compressive imaging (SCI) recovers high-dimensional (3D) data cubes from a single 2D measurement, enabling diverse applications like video and hyperspectral imaging to go beyond standard techniques in terms of acquisition speed and efficiency. In this paper, we focus on SCI recovery algorithms that employ untrained neural networks (UNNs), such as deep image prior (DIP), to model source structure. Such UNN-based methods are appealing as they have the potential of avoiding the computationally intensive retraining required for different source models and different measurement scenarios. We first develop a theoretical framework for characterizing the performance of such UNN-based methods. The theoretical framework, on the one hand, enables us to optimize the parameters of data-modulating masks, and on the other hand, provides a fundamental connection between the number of data frames that can be recovered from a single measurement to the parameters of the untrained NN. We also employ the recently proposed bagged-deep-image-prior (bagged-DIP) idea to develop SCI Bagged Deep Video Prior (SCI-BDVP) algorithms that address the common challenges faced by standard UNN solutions. Our experimental results show that in video SCI our proposed solution achieves state-of-the-art among UNN methods, and in the case of noisy measurements, it even outperforms supervised solutions.