Unified Single-Image and Video Super-Resolution via Denoising Algorithms

TL;DR

This paper introduces a unified super-resolution framework that leverages denoising algorithms, particularly the VBM3D video denoiser, to effectively handle both single-image and video super-resolution without relying on motion estimation.

Contribution

It proposes a simple, robust, and unified super-resolution method based on denoising algorithms, extending the Plug-and-Play-Prior and RED frameworks to both SISR and VSR tasks.

Findings

Achieves high-quality video super-resolution without motion estimation.

Demonstrates competitive performance with fast processing times.

Unifies single-image and video super-resolution under a common denoising-based framework.

Abstract

Single Image Super-Resolution (SISR) aims to recover a high-resolution image from a given low-resolution version of it. Video Super Resolution (VSR) targets series of given images, aiming to fuse them to create a higher resolution outcome. Although SISR and VSR seem to have a lot in common, most SISR algorithms do not have a simple and direct extension to VSR. VSR is considered a more challenging inverse problem, mainly due to its reliance on a sub-pixel accurate motion-estimation, which has no parallel in SISR. Another complication is the dynamics of the video, often addressed by simply generating a single frame instead of a complete output sequence. In this work we suggest a simple and robust super-resolution framework that can be applied to single images and easily extended to video. Our work relies on the observation that denoising of images and videos is well-managed and very effectively treated by a variety of methods. We exploit the Plug-and-Play-Prior framework and the Regularization-by-Denoising (RED) approach that extends it, and show how to use such denoisers in order to handle the SISR and the VSR problems using a unified formulation and framework. This way, we benefit from the effectiveness and efficiency of existing image/video denoising algorithms, while solving much more challenging problems. More specifically, harnessing the VBM3D video denoiser, we obtain a strongly competitive motion-estimation free VSR algorithm, showing tendency to a high-quality output and fast processing.