Synthesizing Anyone, Anywhere, in Any Pose

TL;DR

This paper introduces TriA-GAN, a novel keypoint-guided GAN for synthesizing full human bodies in any pose and setting, addressing challenges like occlusion and complex backgrounds, with applications in editing and pose variation.

Contribution

The paper presents TriA-GAN, a new GAN architecture that improves in-the-wild human figure synthesis using keypoints and a projected GAN approach, requiring less conditional info than previous methods.

Findings

TriA-GAN outperforms previous methods in synthesis quality.

It requires less conditional information, using only keypoints.

The latent space enables text-guided editing of generated figures.

Abstract

We address the task of in-the-wild human figure synthesis, where the primary goal is to synthesize a full body given any region in any image. In-the-wild human figure synthesis has long been a challenging and under-explored task, where current methods struggle to handle extreme poses, occluding objects, and complex backgrounds. Our main contribution is TriA-GAN, a keypoint-guided GAN that can synthesize Anyone, Anywhere, in Any given pose. Key to our method is projected GANs combined with a well-crafted training strategy, where our simple generator architecture can successfully handle the challenges of in-the-wild full-body synthesis. We show that TriA-GAN significantly improves over previous in-the-wild full-body synthesis methods, all while requiring less conditional information for synthesis (keypoints \vs DensePose). Finally, we show that the latent space of TriA-GAN is compatible with standard unconditional editing techniques, enabling text-guided editing of generated human figures.