Multi-scale deep neural networks for real image super-resolution

TL;DR

This paper introduces multi-scale deep neural networks that process images at different scales to improve real-world image super-resolution, especially when the upscaling factor is unknown, demonstrating robustness and competitive performance.

Contribution

The paper proposes two novel multi-scale neural network architectures for real image super-resolution that reduce computational complexity and handle unknown upscaling factors effectively.

Findings

Robustness of MsDNN in unknown upscaling scenarios

Competitive ranking in NTIRE 2019 challenge

Reduced GPU memory usage through multi-scale processing

Abstract

Single image super-resolution (SR) is extremely difficult if the upscaling factors of image pairs are unknown and different from each other, which is common in real image SR. To tackle the difficulty, we develop two multi-scale deep neural networks (MsDNN) in this work. Firstly, due to the high computation complexity in high-resolution spaces, we process an input image mainly in two different downscaling spaces, which could greatly lower the usage of GPU memory. Then, to reconstruct the details of an image, we design a multi-scale residual network (MsRN) in the downscaling spaces based on the residual blocks. Besides, we propose a multi-scale dense network based on the dense blocks to compare with MsRN. Finally, our empirical experiments show the robustness of MsDNN for image SR when the upscaling factor is unknown. According to the preliminary results of NTIRE 2019 image SR challenge, our team (ZXHresearch@fudan) ranks 21-st among all participants. The implementation of MsDNN is released https://github.com/shangqigao/gsq-image-SR