Ear-to-ear Capture of Facial Intrinsics

TL;DR

This paper introduces a hybrid geometric and photometric method for capturing detailed, relightable 3D face models from ear-to-ear, without requiring geometric calibration, by combining multiview stereo and photometric measurements.

Contribution

It presents a novel approach to integrate surface normals and intrinsic textures into seamless, high-resolution 3D face models using uncalibrated multiview stereo and photometric data.

Findings

Produces highly detailed, relightable face models

Achieves seamless stitching of surface normals and textures

Enables photorealistic rendering from any view

Abstract

We present a practical approach to capturing ear-to-ear face models comprising both 3D meshes and intrinsic textures (i.e. diffuse and specular albedo). Our approach is a hybrid of geometric and photometric methods and requires no geometric calibration. Photometric measurements made in a lightstage are used to estimate view dependent high resolution normal maps. We overcome the problem of having a single photometric viewpoint by capturing in multiple poses. We use uncalibrated multiview stereo to estimate a coarse base mesh to which the photometric views are registered. We propose a novel approach to robustly stitching surface normal and intrinsic texture data into a seamless, complete and highly detailed face model. The resulting relightable models provide photorealistic renderings in any view.