Dance Dance Generation: Motion Transfer for Internet Videos

TL;DR

This paper introduces a novel method for transferring body movements onto a target person in videos collected from the internet, enabling realistic motion transfer through personalized generative models.

Contribution

It develops a personalized motion transfer framework using two neural networks to generate realistic videos of a target person mimicking new motions from other videos.

Findings

The model produces photo-realistic images of the target in new poses.

Quantitative and qualitative evaluations show improved realism over baselines.

Subjective tests confirm the effectiveness of the motion transfer.

Abstract

This work presents computational methods for transferring body movements from one person to another with videos collected in the wild. Specifically, we train a personalized model on a single video from the Internet which can generate videos of this target person driven by the motions of other people. Our model is built on two generative networks: a human (foreground) synthesis net which generates photo-realistic imagery of the target person in a novel pose, and a fusion net which combines the generated foreground with the scene (background), adding shadows or reflections as needed to enhance realism. We validate the the efficacy of our proposed models over baselines with qualitative and quantitative evaluations as well as a subjective test.