3D Face From X: Learning Face Shape from Diverse Sources

TL;DR

This paper introduces a method to learn a comprehensive 3D face model by integrating diverse data sources, including scans, images, and RGB-D data, enhancing face shape modeling.

Contribution

It proposes a unified approach to learn 3D face models from multiple data types, bridging the gap between geometric accuracy and data availability.

Findings

Learned a more powerful face model with diverse data sources

Effectively integrated RGB-D data to enhance face shape modeling

Demonstrated improved 3D face reconstruction accuracy

Abstract

We present a novel method to jointly learn a 3D face parametric model and 3D face reconstruction from diverse sources. Previous methods usually learn 3D face modeling from one kind of source, such as scanned data or in-the-wild images. Although 3D scanned data contain accurate geometric information of face shapes, the capture system is expensive and such datasets usually contain a small number of subjects. On the other hand, in-the-wild face images are easily obtained and there are a large number of facial images. However, facial images do not contain explicit geometric information. In this paper, we propose a method to learn a unified face model from diverse sources. Besides scanned face data and face images, we also utilize a large number of RGB-D images captured with an iPhone X to bridge the gap between the two sources. Experimental results demonstrate that with training data from more sources, we can learn a more powerful face model.