A Unified Framework for Compressive Video Recovery from Coded Exposure Techniques

TL;DR

This paper introduces a unified learning-based framework to compare different coded exposure video acquisition techniques, demonstrating that two-measurement methods like C2B outperform single-measurement methods in static scenes, but offer marginal benefits in dynamic scenes.

Contribution

A novel unified deep learning framework that quantitatively and qualitatively compares single and double measurement coded exposure techniques for video recovery.

Findings

C2B outperforms single-measurement methods in static scenes.

Single-measurement methods are nearly as effective in dynamic scenes.

The framework achieves state-of-the-art reconstruction quality across techniques.

Abstract

Several coded exposure techniques have been proposed for acquiring high frame rate videos at low bandwidth. Most recently, a Coded-2-Bucket camera has been proposed that can acquire two compressed measurements in a single exposure, unlike previously proposed coded exposure techniques, which can acquire only a single measurement. Although two measurements are better than one for an effective video recovery, we are yet unaware of the clear advantage of two measurements, either quantitatively or qualitatively. Here, we propose a unified learning-based framework to make such a qualitative and quantitative comparison between those which capture only a single coded image (Flutter Shutter, Pixel-wise coded exposure) and those that capture two measurements per exposure (C2B). Our learning-based framework consists of a shift-variant convolutional layer followed by a fully convolutional deep neural network. Our proposed unified framework achieves the state of the art reconstructions in all three sensing techniques. Further analysis shows that when most scene points are static, the C2B sensor has a significant advantage over acquiring a single pixel-wise coded measurement. However, when most scene points undergo motion, the C2B sensor has only a marginal benefit over the single pixel-wise coded exposure measurement.