Explorable Super Resolution

TL;DR

This paper introduces explorable super resolution, a framework that allows users to explore multiple plausible high-resolution reconstructions of a low-resolution image through an interactive interface, enhancing interpretability and flexibility.

Contribution

It presents a novel module that wraps existing super resolution networks, ensuring their outputs can be explored and precisely match the low-resolution input after downsampling.

Findings

Enables exploration of diverse plausible HR images.

Guarantees SR outputs match LR input when downsampled.

Improves reconstruction error and handles different blur kernels.

Abstract

Single image super resolution (SR) has seen major performance leaps in recent years. However, existing methods do not allow exploring the infinitely many plausible reconstructions that might have given rise to the observed low-resolution (LR) image. These different explanations to the LR image may dramatically vary in their textures and fine details, and may often encode completely different semantic information. In this paper, we introduce the task of explorable super resolution. We propose a framework comprising a graphical user interface with a neural network backend, allowing editing the SR output so as to explore the abundance of plausible HR explanations to the LR input. At the heart of our method is a novel module that can wrap any existing SR network, analytically guaranteeing that its SR outputs would precisely match the LR input, when downsampled. Besides its importance in our setting, this module is guaranteed to decrease the reconstruction error of any SR network it wraps, and can be used to cope with blur kernels that are different from the one the network was trained for. We illustrate our approach in a variety of use cases, ranging from medical imaging and forensics, to graphics.