Dense Pose Transfer
Natalia Neverova, Riza Alp Guler, Iasonas Kokkinos

TL;DR
This paper introduces a novel method combining surface-based pose estimation with neural synthesis to achieve accurate person image pose transfer, outperforming previous methods on fashion datasets.
Contribution
It presents an integrated pipeline that uses dense pose estimation for better conditioning in pose transfer, trained end-to-end with adversarial and perceptual losses.
Findings
Outperforms state-of-the-art generators on DeepFashion and MVC datasets.
Dense pose estimation provides superior conditioning compared to landmarks or masks.
End-to-end training improves synthesis quality and accuracy.
Abstract
In this work we integrate ideas from surface-based modeling with neural synthesis: we propose a combination of surface-based pose estimation and deep generative models that allows us to perform accurate pose transfer, i.e. synthesize a new image of a person based on a single image of that person and the image of a pose donor. We use a dense pose estimation system that maps pixels from both images to a common surface-based coordinate system, allowing the two images to be brought in correspondence with each other. We inpaint and refine the source image intensities in the surface coordinate system, prior to warping them onto the target pose. These predictions are fused with those of a convolutional predictive module through a neural synthesis module allowing for training the whole pipeline jointly end-to-end, optimizing a combination of adversarial and perceptual losses. We show that dense…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Human Pose and Action Recognition · Generative Adversarial Networks and Image Synthesis
