Divergence-Based Adaptive Extreme Video Completion

TL;DR

This paper introduces a novel divergence-based adaptive method for extreme video completion, effectively reconstructing videos from only 1% of pixels by leveraging color-motion estimation and adaptive filtering.

Contribution

It extends an existing extreme image completion algorithm to videos, incorporating a color KL-divergence approach for better sparse data reconstruction.

Findings

Achieved high PSNR in video reconstruction

Validated on 50 videos with positive mean opinion scores

Demonstrated effective adaptation between spatial and temporal filtering

Abstract

Extreme image or video completion, where, for instance, we only retain 1% of pixels in random locations, allows for very cheap sampling in terms of the required pre-processing. The consequence is, however, a reconstruction that is challenging for humans and inpainting algorithms alike. We propose an extension of a state-of-the-art extreme image completion algorithm to extreme video completion. We analyze a color-motion estimation approach based on color KL-divergence that is suitable for extremely sparse scenarios. Our algorithm leverages the estimate to adapt between its spatial and temporal filtering when reconstructing the sparse randomly-sampled video. We validate our results on 50 publicly-available videos using reconstruction PSNR and mean opinion scores.