Multi-attribute Pizza Generator: Cross-domain Attribute Control with Conditional StyleGAN

TL;DR

This paper introduces MPG, a GAN-based framework that synthesizes realistic pizza images with controllable ingredients, view angles, and styles, by extending StyleGAN2 and bridging real and CGI datasets for improved attribute control.

Contribution

The paper presents a novel multi-attribute conditional GAN architecture that effectively controls content, view, and style attributes in image synthesis, including a new CGI dataset and attribute regularization techniques.

Findings

MPG generates photo-realistic pizza images with specified attributes.

The model successfully controls multiple attributes beyond training data.

Bridging real and CGI datasets improves attribute generalization.

Abstract

Multi-attribute conditional image generation is a challenging problem in computervision. We propose Multi-attribute Pizza Generator (MPG), a conditional Generative Neural Network (GAN) framework for synthesizing images from a trichotomy of attributes: content, view-geometry, and implicit visual style. We design MPG by extending the state-of-the-art StyleGAN2, using a new conditioning technique that guides the intermediate feature maps to learn multi-scale multi-attribute entangled representationsof controlling attributes. Because of the complex nature of the multi-attribute image generation problem, we regularize the image generation by predicting the explicit conditioning attributes (ingredients and view). To synthesize a pizza image with view attributesoutside the range of natural training images, we design a CGI pizza dataset PizzaView using 3D pizza models and employ it to train a view attribute regressor to regularize the generation process, bridging the real and CGI training datasets. To verify the efficacy of MPG, we test it on Pizza10, a carefully annotated multi-ingredient pizza image dataset. MPG can successfully generate photo-realistic pizza images with desired ingredients and view attributes, beyond the range of those observed in real-world training data.