Learning Many-to-Many Mapping for Unpaired Real-World Image Super-resolution and Downscaling

TL;DR

This paper introduces SDFlow, a bidirectional model that jointly learns to downscale and super-resolve real-world images without paired data, effectively capturing the mutual dependency between these processes.

Contribution

The paper presents SDFlow, a novel unsupervised model that simultaneously learns many-to-many mappings for image downscaling and super-resolution, considering their mutual dependency.

Findings

SDFlow generates diverse realistic LR and SR images.

It outperforms existing methods quantitatively and qualitatively.

The model effectively decouples content and degradation in latent space.

Abstract

Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a two-stage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.