Decorating Your Own Bedroom: Locally Controlling Image Generation with Generative Adversarial Networks

TL;DR

This paper introduces LoGAN, a novel method for local editing of images generated by GANs, enabling precise control over specific objects and styles within a scene, demonstrated through bedroom customization.

Contribution

LoGAN presents a new approach with content and style modulation operators and a priority mask for fine-grained local image editing in GANs.

Findings

Effective local editing of objects in generated images

Ability to remove, insert, and rotate objects seamlessly

Complete room refurnishing with customized furniture

Abstract

Generative Adversarial Networks (GANs) have made great success in synthesizing high-quality images. However, how to steer the generation process of a well-trained GAN model and customize the output image is much less explored. It has been recently found that modulating the input latent code used in GANs can reasonably alter some variation factors in the output image, but such manipulation usually presents to change the entire image as a whole. In this work, we propose an effective approach, termed as LoGAN, to support local editing of the output image. Concretely, we introduce two operators, i.e., content modulation and style modulation, together with a priority mask to facilitate the precise control of the intermediate generative features. Taking bedroom synthesis as an instance, we are able to seamlessly remove, insert, shift, and rotate the individual objects inside a room. Furthermore, our method can completely clear out a room and then refurnish it with customized furniture and styles. Experimental results show the great potentials of steering the image generation of pre-trained GANs for versatile image editing.