Semantic Map Injected GAN Training for Image-to-Image Translation

TL;DR

This paper introduces a semantic map injection method during GAN training for image-to-image translation, enhancing generalization and categorical preservation without requiring semantic maps at test time.

Contribution

The paper proposes a novel semantic injected training approach for GANs that improves translation quality and categorical consistency in image-to-image translation tasks.

Findings

Improved SSIM, FID, and KID scores with semantic injection.

Enhanced categorical preservation in generated images.

Better generalization of GAN models on CityScapes and RGB-NIR datasets.

Abstract

Image-to-image translation is the recent trend to transform images from one domain to another domain using generative adversarial network (GAN). The existing GAN models perform the training by only utilizing the input and output modalities of transformation. In this paper, we perform the semantic injected training of GAN models. Specifically, we train with original input and output modalities and inject a few epochs of training for translation from input to semantic map. Lets refer the original training as the training for the translation of input image into target domain. The injection of semantic training in the original training improves the generalization capability of the trained GAN model. Moreover, it also preserves the categorical information in a better way in the generated image. The semantic map is only utilized at the training time and is not required at the test time. The experiments are performed using state-of-the-art GAN models over CityScapes and RGB-NIR stereo datasets. We observe the improved performance in terms of the SSIM, FID and KID scores after injecting semantic training as compared to original training.