ReenactNet: Real-time Full Head Reenactment

TL;DR

ReenactNet is a real-time system that transfers head movements, expressions, and gaze from a source to a target in high-fidelity videos, enabling interactive and instant face reenactment on standard hardware.

Contribution

This work introduces a real-time, interactive head reenactment system capable of high-quality video synthesis using only a webcam and a laptop.

Findings

Operates at approximately 20 fps in real time

Runs on a commodity laptop with webcam input

Enables instant, interactive face reenactment

Abstract

Video-to-video synthesis is a challenging problem aiming at learning a translation function between a sequence of semantic maps and a photo-realistic video depicting the characteristics of a driving video. We propose a head-to-head system of our own implementation capable of fully transferring the human head 3D pose, facial expressions and eye gaze from a source to a target actor, while preserving the identity of the target actor. Our system produces high-fidelity, temporally-smooth and photo-realistic synthetic videos faithfully transferring the human time-varying head attributes from the source to the target actor. Our proposed implementation: 1) works in real time ($\sim 20$ fps), 2) runs on a commodity laptop with a webcam as the only input, 3) is interactive, allowing the participant to drive a target person, e.g. a celebrity, politician, etc, instantly by varying their expressions, head pose, and eye gaze, and visualising the synthesised video concurrently.