3D Face Reconstruction by Learning from Synthetic Data

TL;DR

This paper presents a CNN-based method for 3D face reconstruction from a single image, trained on synthetic data, achieving robust results even with extreme expressions and lighting variations.

Contribution

The novel approach trains a deep neural network using synthetic facial images with known geometry, bypassing the need for large annotated real datasets.

Findings

Successfully reconstructs 3D facial geometry from real images

Performs well with extreme facial expressions and lighting conditions

Outperforms previous methods relying on keypoint localization

Abstract

Fast and robust three-dimensional reconstruction of facial geometric structure from a single image is a challenging task with numerous applications. Here, we introduce a learning-based approach for reconstructing a three-dimensional face from a single image. Recent face recovery methods rely on accurate localization of key characteristic points. In contrast, the proposed approach is based on a Convolutional-Neural-Network (CNN) which extracts the face geometry directly from its image. Although such deep architectures outperform other models in complex computer vision problems, training them properly requires a large dataset of annotated examples. In the case of three-dimensional faces, currently, there are no large volume data sets, while acquiring such big-data is a tedious task. As an alternative, we propose to generate random, yet nearly photo-realistic, facial images for which the geometric form is known. The suggested model successfully recovers facial shapes from real images, even for faces with extreme expressions and under various lighting conditions.