SRFlow: Learning the Super-Resolution Space with Normalizing Flow

TL;DR

SRFlow introduces a normalizing flow-based super-resolution method that models the full distribution of high-resolution images conditioned on low-resolution inputs, enabling diverse and photo-realistic outputs while outperforming GAN-based methods.

Contribution

This work is the first to apply normalizing flows to super-resolution, directly modeling the conditional distribution and allowing for diverse, high-quality image predictions.

Findings

Outperforms state-of-the-art GAN approaches in PSNR and perceptual quality.

Enables flexible image manipulation and content transfer.

Effectively models the ill-posed super-resolution problem.

Abstract

Super-resolution is an ill-posed problem, since it allows for multiple predictions for a given low-resolution image. This fundamental fact is largely ignored by state-of-the-art deep learning based approaches. These methods instead train a deterministic mapping using combinations of reconstruction and adversarial losses. In this work, we therefore propose SRFlow: a normalizing flow based super-resolution method capable of learning the conditional distribution of the output given the low-resolution input. Our model is trained in a principled manner using a single loss, namely the negative log-likelihood. SRFlow therefore directly accounts for the ill-posed nature of the problem, and learns to predict diverse photo-realistic high-resolution images. Moreover, we utilize the strong image posterior learned by SRFlow to design flexible image manipulation techniques, capable of enhancing super-resolved images by, e.g., transferring content from other images. We perform extensive experiments on faces, as well as on super-resolution in general. SRFlow outperforms state-of-the-art GAN-based approaches in terms of both PSNR and perceptual quality metrics, while allowing for diversity through the exploration of the space of super-resolved solutions.