Pose-Invariant 3D Face Alignment

TL;DR

This paper introduces a pose-invariant 3D face alignment method that estimates 2D and 3D landmarks and their visibilities across arbitrary poses, outperforming existing techniques on a large all-pose face dataset.

Contribution

It presents a novel algorithm combining a 3D deformable model with a cascaded regressor to handle arbitrary poses and estimate landmark visibilities automatically.

Findings

Achieves superior accuracy on all-pose face datasets.

Effectively estimates 3D landmarks and visibilities.

Outperforms state-of-the-art face alignment methods.

Abstract

Face alignment aims to estimate the locations of a set of landmarks for a given image. This problem has received much attention as evidenced by the recent advancement in both the methodology and performance. However, most of the existing works neither explicitly handle face images with arbitrary poses, nor perform large-scale experiments on non-frontal and profile face images. In order to address these limitations, this paper proposes a novel face alignment algorithm that estimates both 2D and 3D landmarks and their 2D visibilities for a face image with an arbitrary pose. By integrating a 3D deformable model, a cascaded coupled-regressor approach is designed to estimate both the camera projection matrix and the 3D landmarks. Furthermore, the 3D model also allows us to automatically estimate the 2D landmark visibilities via surface normals. We gather a substantially larger collection of all-pose face images to evaluate our algorithm and demonstrate superior performances than the state-of-the-art methods.