Optimized processing order for 3D hole filling in video sequences using frequency selective extrapolation

TL;DR

This paper introduces an optimized processing order for 3D hole filling in video sequences using frequency selective extrapolation, significantly improving reconstruction quality by closing holes from the outer margin to the center.

Contribution

It proposes a novel processing order for 3D extrapolation that enhances video hole filling quality compared to previous methods.

Findings

Reconstruction quality improved by more than 0.5 dB PSNR.

Holes are effectively closed from outer margin to center.

Significant visual gains demonstrated in experiments.

Abstract

A problem often arising in video communication is the reconstruction of missing or distorted areas in a video sequence. Such holes of unavailable pixels may be caused for example by transmission errors of coded video data or undesired objects like logos. In order to close the holes given neighboring available content, a signal extrapolation has to be performed. The best quality can be achieved, if spatial as well as temporal information is used for the reconstruction. However, the question always is in which order to process the extrapolation to obtain the best result. In this paper, an optimized processing order is introduced for improving the extrapolation quality of Three-dimensional Frequency Selective Extrapolation. Using the proposed optimized order, holes in video sequences can be closed from the outer margin to the center, leading to a higher reconstruction quality, and visually noticeable gains of more than 0.5 dB PSNR are possible.