VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresence
Phong Tran, Egor Zakharov, Long-Nhat Ho, Liwen Hu, Adilbek Karmanov,, Aviral Agarwal, McLean Goldwhite, Ariana Bermudez Venegas, Anh Tuan Tran, Hao, Li

TL;DR
VOODOO XP is a real-time, 3D-aware head reenactment method that accurately reproduces expressive facial movements from a single portrait and driver video, enabling immersive VR telepresence without calibration.
Contribution
It introduces a novel cross-reenactment architecture with multi-stage self-supervision for highly expressive, view-consistent facial reenactment from minimal input, suitable for VR applications.
Findings
Achieves state-of-the-art expressiveness and likeness preservation.
Operates in real-time without calibration or fine-tuning.
Successfully integrated into an end-to-end VR telepresence system.
Abstract
We introduce VOODOO XP: a 3D-aware one-shot head reenactment method that can generate highly expressive facial expressions from any input driver video and a single 2D portrait. Our solution is real-time, view-consistent, and can be instantly used without calibration or fine-tuning. We demonstrate our solution on a monocular video setting and an end-to-end VR telepresence system for two-way communication. Compared to 2D head reenactment methods, 3D-aware approaches aim to preserve the identity of the subject and ensure view-consistent facial geometry for novel camera poses, which makes them suitable for immersive applications. While various facial disentanglement techniques have been introduced, cutting-edge 3D-aware neural reenactment techniques still lack expressiveness and fail to reproduce complex and fine-scale facial expressions. We present a novel cross-reenactment architecture…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVirtual Reality Applications and Impacts · Robotics and Automated Systems · Teleoperation and Haptic Systems
MethodsSparse Evolutionary Training
