Unpaired Pose Guided Human Image Generation

TL;DR

This paper introduces an end-to-end generative adversarial network that creates realistic human images from coarse pose sketches without needing paired data, enabling detailed control over appearance and clothing style.

Contribution

It presents a novel unpaired, pose-guided human image generation model that does not require 3D pose fitting or paired datasets, advancing realistic image synthesis.

Findings

Generated images are highly realistic according to perceptual studies

Participants struggle to distinguish fake images from real ones

The model effectively controls clothing style and pose in generated images

Abstract

This paper studies the task of full generative modelling of realistic images of humans, guided only by coarse sketch of the pose, while providing control over the specific instance or type of outfit worn by the user. This is a difficult problem because input and output domain are very different and direct image-to-image translation becomes infeasible. We propose an end-to-end trainable network under the generative adversarial framework, that provides detailed control over the final appearance while not requiring paired training data and hence allows us to forgo the challenging problem of fitting 3D poses to 2D images. The model allows to generate novel samples conditioned on either an image taken from the target domain or a class label indicating the style of clothing (e.g., t-shirt). We thoroughly evaluate the architecture and the contributions of the individual components experimentally. Finally, we show in a large scale perceptual study that our approach can generate realistic looking images and that participants struggle in detecting fake images versus real samples, especially if faces are blurred.