Photo-realistic 360 Head Avatars in the Wild
Stanislaw Szymanowicz, Virginia Estellers, Tadas Baltrusaitis, and Matthew Johnson

TL;DR
This paper introduces a method to create photo-realistic 3D head avatars from 360-degree videos captured with mobile phones, using synthetic data-trained landmark detection for pose estimation, enabling accessible immersive communication.
Contribution
A novel landmark detector trained on synthetic data for accurate camera pose estimation from mobile phone videos of heads, facilitating realistic avatar creation.
Findings
Effective pose estimation from unstructured mobile videos.
High-quality 3D head avatars generated from commodity hardware.
Validation on synthetic and real-world data demonstrates robustness.
Abstract
Delivering immersive, 3D experiences for human communication requires a method to obtain 360 degree photo-realistic avatars of humans. To make these experiences accessible to all, only commodity hardware, like mobile phone cameras, should be necessary to capture the data needed for avatar creation. For avatars to be rendered realistically from any viewpoint, we require training images and camera poses from all angles. However, we cannot rely on there being trackable features in the foreground or background of all images for use in estimating poses, especially from the side or back of the head. To overcome this, we propose a novel landmark detector trained on synthetic data to estimate camera poses from 360 degree mobile phone videos of a human head for use in a multi-stage optimization process which creates a photo-realistic avatar. We perform validation experiments with synthetic data…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Human Pose and Action Recognition · Video Surveillance and Tracking Methods
