Motion-Adapted Three-Dimensional Frequency Selective Extrapolation

TL;DR

This paper introduces a motion-adapted extension of the 3D frequency selective extrapolation algorithm to improve video reconstruction quality by compensating for motion-induced content changes, achieving up to 1.75 dB gains.

Contribution

The paper presents a novel motion-adapted extension to 3D-FSE that effectively handles changing content due to motion, enhancing reconstruction accuracy.

Findings

Achieves up to 1.75 dB improvement in image quality.

Effectively compensates for motion-induced content changes.

Improves video reconstruction fidelity.

Abstract

It has been shown, that high resolution images can be acquired using a low resolution sensor with non-regular sampling. Therefore, post-processing is necessary. In terms of video data, not only the spatial neighborhood can be used to assist the reconstruction, but also the temporal neighborhood. A popular and well performing algorithm for this kind of problem is the three-dimensional frequency selective extrapolation (3D-FSE) for which a motion adapted version is introduced in this paper. This proposed extension solves the problem of changing content within the area considered by the 3D-FSE, which is caused by motion within the sequence. Because of this motion, it may happen that regions are emphasized during the reconstruction that are not present in the original signal within the considered area. By that, false content is introduced into the extrapolated sequence, which affects the resulting image quality negatively. The novel extension, presented in the following, incorporates motion data of the sequence in order to adapt the algorithm accordingly, and compensates changing content, resulting in gains of up to 1.75 dB compared to the existing 3D-FSE.