Monocular Facial Appearance Capture in the Wild

TL;DR

This paper introduces a novel monocular method for capturing detailed facial appearance in unconstrained environments, accurately reconstructing geometry and reflectance properties without controlled lighting or multi-view setups.

Contribution

It presents a new approach that recovers facial surface geometry, diffuse albedo, and specular properties from simple monocular videos in-the-wild, without assumptions on lighting or environment constraints.

Findings

Achieves near-studio quality facial appearance maps from monocular videos

Handles occlusions and visibility explicitly in the reconstruction process

Operates effectively in unconstrained, real-world environments

Abstract

We present a new method for reconstructing the appearance properties of human faces from a lightweight capture procedure in an unconstrained environment. Our method recovers the surface geometry, diffuse albedo, specular intensity and specular roughness from a monocular video containing a simple head rotation in-the-wild. Notably, we make no simplifying assumptions on the environment lighting, and we explicitly take visibility and occlusions into account. As a result, our method can produce facial appearance maps that approach the fidelity of studio-based multi-view captures, but with a far easier and cheaper procedure.