Dynamic Non-Regular Sampling Sensor Using Frequency Selective Reconstruction

TL;DR

This paper introduces a dynamic non-regular sampling sensor that captures high-resolution images temporally and spatially by selectively reading pixels and reconstructing frames, significantly improving image quality over static methods.

Contribution

The paper proposes a novel sensor concept with dynamic pixel readout and a 3D reconstruction algorithm, enabling high spatial and temporal resolution simultaneously.

Findings

Achieves up to 8.55 dB PSNR gain over static strategies.

Outperforms state-of-the-art frame rate up-conversion and super-resolution methods.

Provides a new approach for high-resolution video acquisition with improved reconstruction quality.

Abstract

Both a high spatial and a high temporal resolution of images and videos are desirable in many applications such as entertainment systems, monitoring manufacturing processes, or video surveillance. Due to the limited throughput of pixels per second, however, there is always a trade-off between acquiring sequences with a high spatial resolution at a low temporal resolution or vice versa. In this paper, a modified sensor concept is proposed which is able to acquire both a high spatial and a high temporal resolution. This is achieved by dynamically reading out only a subset of pixels in a non-regular order to obtain a high temporal resolution. A full high spatial resolution is then obtained by performing a subsequent three-dimensional reconstruction of the partially acquired frames. The main benefit of the proposed dynamic readout is that for each frame, different sampling points are available which is advantageous since this information can significantly enhance the reconstruction quality of the proposed reconstruction algorithm. Using the proposed dynamic readout strategy, gains in PSNR of up to 8.55 dB are achieved compared to a static readout strategy. Compared to other state-of-the-art techniques like frame rate up-conversion or super-resolution which are also able to reconstruct sequences with both a high spatial and a high temporal resolution, average gains in PSNR of up to 6.58 dB are possible.