Inner Space Preserving Generative Pose Machine

TL;DR

This paper introduces ISP-GPM, a novel generative model that reposes articulated figures in images while preserving the background, using an interpretable low-dimensional pose descriptor within a multi-stage GAN framework.

Contribution

The paper proposes a new model that effectively reposes articulated figures in images with background preservation, extending pose manipulation to more complex, real-world scenarios.

Findings

Achieved over 80% accuracy on PCK0.5 pose estimation metric.

Successfully preserved background details during reposing.

Effectively handled highly articulated human figures with diverse poses.

Abstract

Image-based generative methods, such as generative adversarial networks (GANs) have already been able to generate realistic images with much context control, specially when they are conditioned. However, most successful frameworks share a common procedure which performs an image-to-image translation with pose of figures in the image untouched. When the objective is reposing a figure in an image while preserving the rest of the image, the state-of-the-art mainly assumes a single rigid body with simple background and limited pose shift, which can hardly be extended to the images under normal settings. In this paper, we introduce an image "inner space" preserving model that assigns an interpretable low-dimensional pose descriptor (LDPD) to an articulated figure in the image. Figure reposing is then generated by passing the LDPD and the original image through multi-stage augmented hourglass networks in a conditional GAN structure, called inner space preserving generative pose machine (ISP-GPM). We evaluated ISP-GPM on reposing human figures, which are highly articulated with versatile variations. Test of a state-of-the-art pose estimator on our reposed dataset gave an accuracy over 80% on PCK0.5 metric. The results also elucidated that our ISP-GPM is able to preserve the background with high accuracy while reasonably recovering the area blocked by the figure to be reposed.