Temporal Kernel Consistency for Blind Video Super-Resolution

TL;DR

This paper explores the importance of temporal kernel consistency in blind video super-resolution, demonstrating that leveraging kernel stability across frames improves restoration quality and achieves state-of-the-art results.

Contribution

It introduces the concept of kernel temporal consistency in blind video SR and adapts existing methods to exploit this property for better performance.

Findings

Kernel estimates vary across frames depending on scene dynamics.

Using fixed kernels can cause visual artifacts in video SR.

Leveraging kernel consistency improves both quantitative and qualitative results.

Abstract

Deep learning-based blind super-resolution (SR) methods have recently achieved unprecedented performance in upscaling frames with unknown degradation. These models are able to accurately estimate the unknown downscaling kernel from a given low-resolution (LR) image in order to leverage the kernel during restoration. Although these approaches have largely been successful, they are predominantly image-based and therefore do not exploit the temporal properties of the kernels across multiple video frames. In this paper, we investigated the temporal properties of the kernels and highlighted its importance in the task of blind video super-resolution. Specifically, we measured the kernel temporal consistency of real-world videos and illustrated how the estimated kernels might change per frame in videos of varying dynamicity of the scene and its objects. With this new insight, we revisited previous popular video SR approaches, and showed that previous assumptions of using a fixed kernel throughout the restoration process can lead to visual artifacts when upscaling real-world videos. In order to counteract this, we tailored existing single-image and video SR techniques to leverage kernel consistency during both kernel estimation and video upscaling processes. Extensive experiments on synthetic and real-world videos show substantial restoration gains quantitatively and qualitatively, achieving the new state-of-the-art in blind video SR and underlining the potential of exploiting kernel temporal consistency.