Single-Camera 3D Head Fitting for Mixed Reality Clinical Applications
Tejas Mane, Aylar Bayramova, Kostas Daniilidis, Philippos Mordohai,, Elena Bernardis

TL;DR
This paper presents a novel method for reconstructing complete 3D head shapes from a single video camera, enabling accurate mixed reality applications by combining dense reconstruction with a two-stage fitting process.
Contribution
It introduces a new two-stage fitting process that combines dense 3D reconstruction with a morphable model to accurately recover full head geometry from monocular video.
Findings
Successfully reconstructs consistent 3D head shapes across diverse environments.
Works with videos from different smartphones and varying poses.
Enables future mixed reality applications with detailed head models.
Abstract
We address the problem of estimating the shape of a person's head, defined as the geometry of the complete head surface, from a video taken with a single moving camera, and determining the alignment of the fitted 3D head for all video frames, irrespective of the person's pose. 3D head reconstructions commonly tend to focus on perfecting the face reconstruction, leaving the scalp to a statistical approximation. Our goal is to reconstruct the head model of each person to enable future mixed reality applications. To do this, we recover a dense 3D reconstruction and camera information via structure-from-motion and multi-view stereo. These are then used in a new two-stage fitting process to recover the 3D head shape by iteratively fitting a 3D morphable model of the head with the dense reconstruction in canonical space and fitting it to each person's head, using both traditional facial…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
