MC-GAN: Multi-conditional Generative Adversarial Network for Image Synthesis

TL;DR

This paper introduces MC-GAN, a multi-conditional GAN that generates object images from text attributes while preserving background details, advancing text-to-image synthesis with background control.

Contribution

It presents a novel multi-conditional GAN with a synthesis block that disentangles object and background, enabling realistic image generation with background preservation.

Findings

Generates photo-realistic images at 128x128 resolution.

Successfully controls background and object attributes jointly.

Demonstrates effectiveness on bird and flower datasets.

Abstract

In this paper, we introduce a new method for generating an object image from text attributes on a desired location, when the base image is given. One step further to the existing studies on text-to-image generation mainly focusing on the object's appearance, the proposed method aims to generate an object image preserving the given background information, which is the first attempt in this field. To tackle the problem, we propose a multi-conditional GAN (MC-GAN) which controls both the object and background information jointly. As a core component of MC-GAN, we propose a synthesis block which disentangles the object and background information in the training stage. This block enables MC-GAN to generate a realistic object image with the desired background by controlling the amount of the background information from the given base image using the foreground information from the text attributes. From the experiments with Caltech-200 bird and Oxford-102 flower datasets, we show that our model is able to generate photo-realistic images with a resolution of 128 x 128. The source code of MC-GAN is released.