Novel Consistency Check For Fast Recursive Reconstruction Of Non-Regularly Sampled Video Data

TL;DR

This paper introduces a faster, more accurate recursive reconstruction method for non-regularly sampled video data from novel quarter sampling sensors, significantly improving quality and efficiency over existing techniques.

Contribution

It proposes a novel recursive frequency selective reconstruction method with enhanced consistency checking that handles dynamic masks and reduces computational complexity.

Findings

Reconstruction quality improved by +1.01 dB over state-of-the-art.

Achieves about +1.52 dB gain over single frame reconstruction with dynamic masks.

Reduces consistency check complexity by a factor of 13.

Abstract

Quarter sampling is a novel sensor design that allows for an acquisition of higher resolution images without increasing the number of pixels. When being used for video data, one out of four pixels is measured in each frame. Effectively, this leads to a non-regular spatio-temporal sub-sampling. Compared to purely spatial or temporal sub-sampling, this allows for an increased reconstruction quality, as aliasing artifacts can be reduced. For the fast reconstruction of such sensor data with a fixed mask, recursive variant of frequency selective reconstruction (FSR) was proposed. Here, pixels measured in previous frames are projected into the current frame to support its reconstruction. In doing so, the motion between the frames is computed using template matching. Since some of the motion vectors may be erroneous, it is important to perform a proper consistency checking. In this paper, we propose faster consistency checking methods as well as a novel recursive FSR that uses the projected pixels different than in literature and can handle dynamic masks. Altogether, we are able to significantly increase the reconstruction quality by + 1.01 dB compared to the state-of-the-art recursive reconstruction method using a fixed mask. Compared to a single frame reconstruction, an average gain of about + 1.52 dB is achieved for dynamic masks. At the same time, the computational complexity of the consistency checks is reduced by a factor of 13 compared to the literature algorithm.