Inpainting-based Video Compression in FullHD

TL;DR

This paper introduces a real-time, inpainting-based video compression method for FullHD videos using PDEs, outperforming previous approaches in quality and speed with a CPU implementation.

Contribution

It presents a novel PDE-based architecture for inpainting video compression that achieves real-time FullHD decoding on CPU, a significant advancement over prior methods.

Findings

Achieves real-time FullHD video decoding on CPU

Outperforms previous inpainting-based codecs in quality and speed

Introduces efficient PDE-based prediction and correction techniques

Abstract

Compression methods based on inpainting are an evolving alternative to classical transform-based codecs for still images. Attempts to apply these ideas to video compression are rare, since reaching real-time performance is very challenging. Therefore, current approaches focus on simplified frame-by-frame reconstructions that ignore temporal redundancies. As a remedy, we propose a highly efficient, real-time capable prediction and correction approach that fully relies on partial differential equations (PDEs) in all steps of the codec: Dense variational optic flow fields yield accurate motion-compensated predictions, while homogeneous diffusion inpainting is applied for intra prediction. To compress residuals, we introduce a new highly efficient block-based variant of pseudodifferential inpainting. Our novel architecture outperforms other inpainting-based video codecs in terms of both quality and speed. For the first time in inpainting-based video compression, we can decompress FullHD (1080p) videos in real-time with a fully CPU-based implementation, outperforming previous approaches by roughly one order of magnitude.