Automatic Face Reenactment
Pablo Garrido, Levi Valgaerts, Ole Rehmsen, Thorsten Thormaehlen,, Patrick Perez, Christian Theobalt

TL;DR
This paper introduces an automatic, image-based facial reenactment system that replaces an actor's face in a video with a user's face from a short source video, maintaining the original performance without needing a 3D model.
Contribution
It presents a novel reenactment pipeline combining image retrieval with appearance and motion matching, enabling convincing results from short, in-the-wild source videos without complex modeling.
Findings
System produces realistic reenactments from short source videos.
It is robust to head motion and does not require source and target to be similar.
Effective on low-quality and internet-sourced footage.
Abstract
We propose an image-based, facial reenactment system that replaces the face of an actor in an existing target video with the face of a user from a source video, while preserving the original target performance. Our system is fully automatic and does not require a database of source expressions. Instead, it is able to produce convincing reenactment results from a short source video captured with an off-the-shelf camera, such as a webcam, where the user performs arbitrary facial gestures. Our reenactment pipeline is conceived as part image retrieval and part face transfer: The image retrieval is based on temporal clustering of target frames and a novel image matching metric that combines appearance and motion to select candidate frames from the source video, while the face transfer uses a 2D warping strategy that preserves the user's identity. Our system excels in simplicity as it does…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
