Everybody Dance Now
Caroline Chan, Shiry Ginosar, Tinghui Zhou, Alexei A. Efros

TL;DR
This paper introduces a simple yet effective method for transferring dance motions from a source to a target person using pose-based video translation, along with a forensics tool and an open dataset.
Contribution
It presents a novel pose-to-appearance translation approach for motion transfer, a face synthesis pipeline, and releases a new dataset for training and evaluation.
Findings
Produces compelling dance motion transfer results
Includes a forensics tool to detect synthetic videos
Provides an open-source dataset for training
Abstract
This paper presents a simple method for "do as I do" motion transfer: given a source video of a person dancing, we can transfer that performance to a novel (amateur) target after only a few minutes of the target subject performing standard moves. We approach this problem as video-to-video translation using pose as an intermediate representation. To transfer the motion, we extract poses from the source subject and apply the learned pose-to-appearance mapping to generate the target subject. We predict two consecutive frames for temporally coherent video results and introduce a separate pipeline for realistic face synthesis. Although our method is quite simple, it produces surprisingly compelling results (see video). This motivates us to also provide a forensics tool for reliable synthetic content detection, which is able to distinguish videos synthesized by our system from real data. In…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Everybody Dance Now! - AI-Based Motion Transfer· youtube
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Face recognition and analysis · Human Pose and Action Recognition
