Neural Head Avatars from Monocular RGB Videos
Philip-William Grassal (1), Malte Prinzler (1), Titus Leistner (1),, Carsten Rother (1), Matthias Nie{\ss}ner (2), Justus Thies (3) ((1), Heidelberg University, (2) Technical University of Munich, (3) Max Planck, Institute for Intelligent Systems)

TL;DR
This paper introduces Neural Head Avatars, a neural representation learned from monocular RGB videos that models the shape and appearance of human heads for realistic avatar synthesis in AR/VR and entertainment.
Contribution
It proposes a hybrid model combining a morphable shape model with neural networks to produce detailed, animatable head avatars from monocular videos, outperforming prior methods.
Findings
Accurately extrapolates to unseen poses and views
Generates natural expressions with sharp textures
Outperforms state-of-the-art in quality and synthesis
Abstract
We present Neural Head Avatars, a novel neural representation that explicitly models the surface geometry and appearance of an animatable human avatar that can be used for teleconferencing in AR/VR or other applications in the movie or games industry that rely on a digital human. Our representation can be learned from a monocular RGB portrait video that features a range of different expressions and views. Specifically, we propose a hybrid representation consisting of a morphable model for the coarse shape and expressions of the face, and two feed-forward networks, predicting vertex offsets of the underlying mesh as well as a view- and expression-dependent texture. We demonstrate that this representation is able to accurately extrapolate to unseen poses and view points, and generates natural expressions while providing sharp texture details. Compared to previous works on head avatars,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
Topics3D Shape Modeling and Analysis · Face recognition and analysis · Generative Adversarial Networks and Image Synthesis
