On the Diversity of Realistic Image Synthesis

TL;DR

This paper introduces a novel diversity loss for image synthesis models, enabling the generation of more varied and controllable realistic images from semantic layouts without compromising image quality.

Contribution

The paper proposes a new diversity loss that enhances output variability and user control in image synthesis, addressing limitations of existing methods.

Findings

Generated images are significantly more diverse.

Diversity does not compromise image realism.

User manipulation of outputs is enabled.

Abstract

Many image processing tasks can be formulated as translating images between two image domains, such as colorization, super resolution and conditional image synthesis. In most of these tasks, an input image may correspond to multiple outputs. However, current existing approaches only show very minor diversity of the outputs. In this paper, we present a novel approach to synthesize diverse realistic images corresponding to a semantic layout. We introduce a diversity loss objective, which maximizes the distance between synthesized image pairs and links the input noise to the semantic segments in the synthesized images. Thus, our approach can not only produce diverse images, but also allow users to manipulate the output images by adjusting the noise manually. Experimental results show that images synthesized by our approach are significantly more diverse than that of the current existing works and equipping our diversity loss does not degrade the reality of the base networks.