Face Alignment in Full Pose Range: A 3D Total Solution

TL;DR

This paper introduces 3D Dense Face Alignment (3DDFA), a novel framework that effectively aligns faces across the full pose range up to 90 degrees by fitting a dense 3D Morphable Model using CNNs and synthesizing training data.

Contribution

The paper presents a new 3D face alignment method that handles large poses, overcoming landmark visibility issues and appearance variations with a dense 3D model and data synthesis.

Findings

Significant improvement over state-of-the-art on AFLW database

Effective handling of large pose variations up to 90 degrees

Utilization of 3D face synthesis for training enhancement

Abstract

Face alignment, which fits a face model to an image and extracts the semantic meanings of facial pixels, has been an important topic in the computer vision community. However, most algorithms are designed for faces in small to medium poses (yaw angle is smaller than 45 degrees), which lack the ability to align faces in large poses up to 90 degrees. The challenges are three-fold. Firstly, the commonly used landmark face model assumes that all the landmarks are visible and is therefore not suitable for large poses. Secondly, the face appearance varies more drastically across large poses, from the frontal view to the profile view. Thirdly, labelling landmarks in large poses is extremely challenging since the invisible landmarks have to be guessed. In this paper, we propose to tackle these three challenges in an new alignment framework termed 3D Dense Face Alignment (3DDFA), in which a dense 3D Morphable Model (3DMM) is fitted to the image via Cascaded Convolutional Neural Networks. We also utilize 3D information to synthesize face images in profile views to provide abundant samples for training. Experiments on the challenging AFLW database show that the proposed approach achieves significant improvements over the state-of-the-art methods.