Be Your Own Prada: Fashion Synthesis with Structural Coherence

TL;DR

This paper introduces a two-stage generative adversarial network that creates new clothing on a person in an image based on language descriptions, ensuring structural coherence and precise region generation.

Contribution

It proposes a novel two-stage framework with semantic segmentation and compositional mapping for fashion synthesis conditioned on language descriptions.

Findings

Effective generation of clothing with structural coherence.

High-quality visual results demonstrated through evaluations.

User study confirms realism and relevance of generated images.

Abstract

We present a novel and effective approach for generating new clothing on a wearer through generative adversarial learning. Given an input image of a person and a sentence describing a different outfit, our model "redresses" the person as desired, while at the same time keeping the wearer and her/his pose unchanged. Generating new outfits with precise regions conforming to a language description while retaining wearer's body structure is a new challenging task. Existing generative adversarial networks are not ideal in ensuring global coherence of structure given both the input photograph and language description as conditions. We address this challenge by decomposing the complex generative process into two conditional stages. In the first stage, we generate a plausible semantic segmentation map that obeys the wearer's pose as a latent spatial arrangement. An effective spatial constraint is formulated to guide the generation of this semantic segmentation map. In the second stage, a generative model with a newly proposed compositional mapping layer is used to render the final image with precise regions and textures conditioned on this map. We extended the DeepFashion dataset [8] by collecting sentence descriptions for 79K images. We demonstrate the effectiveness of our approach through both quantitative and qualitative evaluations. A user study is also conducted. The codes and the data are available at http://mmlab.ie.cuhk. edu.hk/projects/FashionGAN/.