Few-Shot Human Motion Transfer by Personalized Geometry and Texture Modeling

TL;DR

This paper introduces a novel few-shot human motion transfer method that personalizes geometry and texture modeling to generate realistic human images with minimal appearance inputs, outperforming existing approaches.

Contribution

It proposes a personalized UV map generation framework combining shape and pose information, with fine-tuning of textures at test time to enhance quality without overfitting.

Findings

Outperforms state-of-the-art methods qualitatively and quantitatively

Effective personalization of geometry and texture from few source images

Test-time fine-tuning improves texture quality without artifacts

Abstract

We present a new method for few-shot human motion transfer that achieves realistic human image generation with only a small number of appearance inputs. Despite recent advances in single person motion transfer, prior methods often require a large number of training images and take long training time. One promising direction is to perform few-shot human motion transfer, which only needs a few of source images for appearance transfer. However, it is particularly challenging to obtain satisfactory transfer results. In this paper, we address this issue by rendering a human texture map to a surface geometry (represented as a UV map), which is personalized to the source person. Our geometry generator combines the shape information from source images, and the pose information from 2D keypoints to synthesize the personalized UV map. A texture generator then generates the texture map conditioned on the texture of source images to fill out invisible parts. Furthermore, we may fine-tune the texture map on the manifold of the texture generator from a few source images at the test time, which improves the quality of the texture map without over-fitting or artifacts. Extensive experiments show the proposed method outperforms state-of-the-art methods both qualitatively and quantitatively. Our code is available at https://github.com/HuangZhiChao95/FewShotMotionTransfer.