Instance Map based Image Synthesis with a Denoising Generative Adversarial Network

TL;DR

This paper introduces a denoising GAN framework for semantic layout-based image synthesis, improving image quality and overlapped object generation by incorporating a one-hot semantic map and enhanced loss functions.

Contribution

The paper proposes a novel denoising GAN approach with a one-hot semantic label map and improved loss functions to enhance image synthesis quality and overlapped object generation.

Findings

Effective synthesis on Cityscapes, Facades, NYU datasets

Improved image quality and object handling

Demonstrated superiority over existing methods

Abstract

Semantic layouts based Image synthesizing, which has benefited from the success of Generative Adversarial Network (GAN), has drawn much attention in these days. How to enhance the synthesis image equality while keeping the stochasticity of the GAN is still a challenge. We propose a novel denoising framework to handle this problem. The overlapped objects generation is another challenging task when synthesizing images from a semantic layout to a realistic RGB photo. To overcome this deficiency, we include a one-hot semantic label map to force the generator paying more attention on the overlapped objects generation. Furthermore, we improve the loss function of the discriminator by considering perturb loss and cascade layer loss to guide the generation process. We applied our methods on the Cityscapes, Facades and NYU datasets and demonstrate the image generation ability of our model.