Efficient High-Resolution Image-to-Image Translation using Multi-Scale Gradient U-Net

TL;DR

This paper introduces a Multi-Scale Gradient U-Net that enables high-resolution image-to-image translation up to 2048x1024, achieving photo-realistic results efficiently compared to existing methods.

Contribution

It proposes a novel MSG U-Net architecture with multi-scale gradient flow from multiple discriminators, improving high-resolution translation quality and efficiency.

Findings

Achieves photo-realistic high-resolution translation up to 2048x1024.

Reduces inference time by approximately 2.5 times compared to Pix2Pix-HD.

Demonstrates effective multi-scale gradient flow for high-quality image synthesis.

Abstract

Recently, Conditional Generative Adversarial Network (Conditional GAN) have shown very promising performance in several image-to-image translation applications. However, the uses of these conditional GANs are quite limited to low-resolution images, such as 256X256.The Pix2Pix-HD is a recent attempt to utilize the conditional GAN for high-resolution image synthesis. In this paper, we propose a Multi-Scale Gradient based U-Net (MSG U-Net) model for high-resolution image-to-image translation up to 2048X1024 resolution. The proposed model is trained by allowing the flow of gradients from multiple-discriminators to a single generator at multiple scales. The proposed MSG U-Net architecture leads to photo-realistic high-resolution image-to-image translation. Moreover, the proposed model is computationally efficient as com-pared to the Pix2Pix-HD with an improvement in the inference time nearly by 2.5 times. We provide the code of MSG U-Net model at https://github.com/laxmaniron/MSG-U-Net.