One-to-many Approach for Improving Super-Resolution

TL;DR

This paper enhances GAN-based super-resolution by incorporating one-to-many distribution modeling, improving perceptual quality and achieving state-of-the-art results in extreme super-resolution tasks.

Contribution

It adapts SRFlow's one-to-many approach to GANs, modifies the generator for distribution estimation, and introduces training techniques to boost perceptual quality.

Findings

Improved ESRGAN performance in x4 perceptual super-resolution.

Achieved state-of-the-art LPIPS score in x16 extreme super-resolution.

Enhanced perceptual quality of generated images.

Abstract

Recently, there has been discussions on the ill-posed nature of super-resolution that multiple possible reconstructions exist for a given low-resolution image. Using normalizing flows, SRflow[23] achieves state-of-the-art perceptual quality by learning the distribution of the output instead of a deterministic output to one estimate. In this paper, we adapt the concepts of SRFlow to improve GAN-based super-resolution by properly implementing the one-to-many property. We modify the generator to estimate a distribution as a mapping from random noise. We improve the content loss that hampers the perceptual training objectives. We also propose additional training techniques to further enhance the perceptual quality of generated images. Using our proposed methods, we were able to improve the performance of ESRGAN[1] in x4 perceptual SR and achieve the state-of-the-art LPIPS score in x16 perceptual extreme SR by applying our methods to RFB-ESRGAN[21].