3D Face Morphable Models "In-the-Wild"

TL;DR

This paper introduces the first 3D Morphable Model capable of reconstructing faces in unconstrained, real-world conditions by combining a shape model with an 'in-the-wild' texture model, simplifying fitting and improving performance.

Contribution

It presents the first 'in-the-wild' 3DMM that integrates shape and texture models for unconstrained facial reconstruction, along with a fast fitting algorithm and a new 3D facial database.

Findings

Achieved state-of-the-art quantitative reconstruction performance.

Demonstrated qualitative results on standard 'in-the-wild' datasets.

Released an open-source implementation as part of the Menpo Project.

Abstract

3D Morphable Models (3DMMs) are powerful statistical models of 3D facial shape and texture, and among the state-of-the-art methods for reconstructing facial shape from single images. With the advent of new 3D sensors, many 3D facial datasets have been collected containing both neutral as well as expressive faces. However, all datasets are captured under controlled conditions. Thus, even though powerful 3D facial shape models can be learnt from such data, it is difficult to build statistical texture models that are sufficient to reconstruct faces captured in unconstrained conditions ("in-the-wild"). In this paper, we propose the first, to the best of our knowledge, "in-the-wild" 3DMM by combining a powerful statistical model of facial shape, which describes both identity and expression, with an "in-the-wild" texture model. We show that the employment of such an "in-the-wild" texture model greatly simplifies the fitting procedure, because there is no need to optimize with regards to the illumination parameters. Furthermore, we propose a new fast algorithm for fitting the 3DMM in arbitrary images. Finally, we have captured the first 3D facial database with relatively unconstrained conditions and report quantitative evaluations with state-of-the-art performance. Complementary qualitative reconstruction results are demonstrated on standard "in-the-wild" facial databases. An open source implementation of our technique is released as part of the Menpo Project.