Sampling-Priors-Augmented Deep Unfolding Network for Robust Video Compressive Sensing

TL;DR

This paper introduces SPA-DUN, a lightweight deep unfolding network that uses sampling priors to achieve robust, efficient, and generalizable video compressed sensing reconstruction across various sampling settings.

Contribution

The paper proposes SPA-DUN, a novel deep unfolding network that incorporates sampling priors for robustness and efficiency in VCS, enabling a single model to handle multiple sampling scenarios.

Findings

Achieves state-of-the-art performance on simulation and real datasets.

Handles arbitrary sampling settings with a single model.

Demonstrates high efficiency and robustness in VCS reconstruction.

Abstract

Video Compressed Sensing (VCS) aims to reconstruct multiple frames from one single captured measurement, thus achieving high-speed scene recording with a low-frame-rate sensor. Although there have been impressive advances in VCS recently, those state-of-the-art (SOTA) methods also significantly increase model complexity and suffer from poor generality and robustness, which means that those networks need to be retrained to accommodate the new system. Such limitations hinder the real-time imaging and practical deployment of models. In this work, we propose a Sampling-Priors-Augmented Deep Unfolding Network (SPA-DUN) for efficient and robust VCS reconstruction. Under the optimization-inspired deep unfolding framework, a lightweight and efficient U-net is exploited to downsize the model while improving overall performance. Moreover, the prior knowledge from the sampling model is utilized to dynamically modulate the network features to enable single SPA-DUN to handle arbitrary sampling settings, augmenting interpretability and generality. Extensive experiments on both simulation and real datasets demonstrate that SPA-DUN is not only applicable for various sampling settings with one single model but also achieves SOTA performance with incredible efficiency.