VCISR: Blind Single Image Super-Resolution with Video Compression Synthetic Data

TL;DR

This paper introduces a novel video compression-based degradation model for blind single image super-resolution, enabling neural networks to effectively restore images degraded by video compression artifacts, improving quality and generalization.

Contribution

The work presents the first synthesis method incorporating video compression artifacts into training data for blind SISR, enhancing the restoration of video compression degradations.

Findings

Achieves state-of-the-art results in no-reference image quality assessment.

Demonstrates superior visual quality on various datasets.

Performs comparably or better than dedicated video super-resolution methods.

Abstract

In the blind single image super-resolution (SISR) task, existing works have been successful in restoring image-level unknown degradations. However, when a single video frame becomes the input, these works usually fail to address degradations caused by video compression, such as mosquito noise, ringing, blockiness, and staircase noise. In this work, we for the first time, present a video compression-based degradation model to synthesize low-resolution image data in the blind SISR task. Our proposed image synthesizing method is widely applicable to existing image datasets, so that a single degraded image can contain distortions caused by the lossy video compression algorithms. This overcomes the leak of feature diversity in video data and thus retains the training efficiency. By introducing video coding artifacts to SISR degradation models, neural networks can super-resolve images with the ability to restore video compression degradations, and achieve better results on restoring generic distortions caused by image compression as well. Our proposed approach achieves superior performance in SOTA no-reference Image Quality Assessment, and shows better visual quality on various datasets. In addition, we evaluate the SISR neural network trained with our degradation model on video super-resolution (VSR) datasets. Compared to architectures specifically designed for the VSR purpose, our method exhibits similar or better performance, evidencing that the presented strategy on infusing video-based degradation is generalizable to address more complicated compression artifacts even without temporal cues.