EMOCA: Emotion Driven Monocular Face Capture and Animation
Radek Danecek, Michael J. Black, Timo Bolkart

TL;DR
EMOCA introduces a novel emotion consistency loss in 3D face reconstruction from monocular images, significantly improving the emotional fidelity of facial expressions while maintaining competitive geometric accuracy.
Contribution
The paper proposes a new deep perceptual emotion consistency loss for monocular 3D face reconstruction, enhancing emotional expression accuracy beyond existing methods.
Findings
Outperforms existing methods in emotional expression quality
Achieves comparable geometric reconstruction errors
Effective in in-the-wild emotion recognition
Abstract
As 3D facial avatars become more widely used for communication, it is critical that they faithfully convey emotion. Unfortunately, the best recent methods that regress parametric 3D face models from monocular images are unable to capture the full spectrum of facial expression, such as subtle or extreme emotions. We find the standard reconstruction metrics used for training (landmark reprojection error, photometric error, and face recognition loss) are insufficient to capture high-fidelity expressions. The result is facial geometries that do not match the emotional content of the input image. We address this with EMOCA (EMOtion Capture and Animation), by introducing a novel deep perceptual emotion consistency loss during training, which helps ensure that the reconstructed 3D expression matches the expression depicted in the input image. While EMOCA achieves 3D reconstruction errors that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Face and Expression Recognition · Facial Nerve Paralysis Treatment and Research
