PAV: Personalized Head Avatar from Unstructured Video Collection
Akin Caliskan, Berkay Kicanaoglu, Hyeongwoo Kim

TL;DR
PAV introduces a novel dynamic deformable neural radiance field that models multi-appearance and shape variations of a human head from unstructured video collections, enabling high-quality face synthesis from arbitrary viewpoints and expressions.
Contribution
The paper presents the first unified NeRF framework capable of modeling appearance and shape variations for the same subject from monocular videos.
Findings
Outperforms baseline in rendering quality.
Effectively models appearance and shape changes.
Handles multi-appearance head synthesis.
Abstract
We propose PAV, Personalized Head Avatar for the synthesis of human faces under arbitrary viewpoints and facial expressions. PAV introduces a method that learns a dynamic deformable neural radiance field (NeRF), in particular from a collection of monocular talking face videos of the same character under various appearance and shape changes. Unlike existing head NeRF methods that are limited to modeling such input videos on a per-appearance basis, our method allows for learning multi-appearance NeRFs, introducing appearance embedding for each input video via learnable latent neural features attached to the underlying geometry. Furthermore, the proposed appearance-conditioned density formulation facilitates the shape variation of the character, such as facial hair and soft tissues, in the radiance field prediction. To the best of our knowledge, our approach is the first dynamic deformable…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Motion and Animation · Human Pose and Action Recognition · Generative Adversarial Networks and Image Synthesis
