Deep learning architectural designs for super-resolution of noisy images

TL;DR

This paper explores two deep learning architectures for super-resolution of noisy images, demonstrating that one excels with known noise types while the other performs better with unseen noise, advancing super-resolution in uncontrolled scenarios.

Contribution

It introduces and compares in-network and pre-network architectures for joint denoising and super-resolution, highlighting their respective advantages in different noise conditions.

Findings

In-network design performs best with known noise types.

Pre-network design outperforms on unseen noise types.

Both architectures improve super-resolution of noisy images.

Abstract

Recent advances in deep learning have led to significant improvements in single image super-resolution (SR) research. However, due to the amplification of noise during the upsampling steps, state-of-the-art methods often fail at reconstructing high-resolution images from noisy versions of their low-resolution counterparts. However, this is especially important for images from unknown cameras with unseen types of image degradation. In this work, we propose to jointly perform denoising and super-resolution. To this end, we investigate two architectural designs: "in-network" combines both tasks at feature level, while "pre-network" first performs denoising and then super-resolution. Our experiments show that both variants have specific advantages: The in-network design obtains the strongest results when the type of image corruption is aligned in the training and testing dataset, for any choice of denoiser. The pre-network design exhibits superior performance on unseen types of image corruption, which is a pathological failure case of existing super-resolution models. We hope that these findings help to enable super-resolution also in less constrained scenarios where source camera or imaging conditions are not well controlled. Source code and pretrained models are available at https://github.com/ angelvillar96/super-resolution-noisy-images.