Multi-Modality Image Super-Resolution using Generative Adversarial Networks

TL;DR

This paper introduces two GAN-based models for joint multi-modality image super-resolution and translation, aiming to recover high-resolution images in one modality from low-resolution images in another, with promising results on day/night images.

Contribution

The paper proposes two novel GAN models for simultaneous multi-modality image super-resolution and translation, addressing a joint problem not previously explored in this manner.

Findings

Models achieve promising qualitative results

Models outperform baseline methods in quantitative metrics

Effective recovery of high-resolution day images from low-resolution night images

Abstract

Over the past few years deep learning-based techniques such as Generative Adversarial Networks (GANs) have significantly improved solutions to image super-resolution and image-to-image translation problems. In this paper, we propose a solution to the joint problem of image super-resolution and multi-modality image-to-image translation. The problem can be stated as the recovery of a high-resolution image in a modality, given a low-resolution observation of the same image in an alternative modality. Our paper offers two models to address this problem and will be evaluated on the recovery of high-resolution day images given low-resolution night images of the same scene. Promising qualitative and quantitative results will be presented for each model.