# Face Alignment using a 3D Deeply-initialized Ensemble of Regression Trees

## TL;DR

This paper introduces 3DDE, a robust face alignment method that combines 3D face modeling with ensemble regression trees, improving accuracy under occlusions, pose variations, and ambiguous configurations.

## Contribution

The paper presents a novel 3D face model initialization and a coarse-to-fine ensemble of regression trees for improved face alignment accuracy.

## Key findings

- 3DDE outperforms state-of-the-art on multiple benchmarks.
- Addresses occlusions and large pose variations effectively.
- Reveals dataset bias through cross-dataset experiments.

## Abstract

Face alignment algorithms locate a set of landmark points in images of faces taken in unrestricted situations. State-of-the-art approaches typically fail or lose accuracy in the presence of occlusions, strong deformations, large pose variations and ambiguous configurations. In this paper we present 3DDE, a robust and efficient face alignment algorithm based on a coarse-to-fine cascade of ensembles of regression trees. It is initialized by robustly fitting a 3D face model to the probability maps produced by a convolutional neural network. With this initialization we address self-occlusions and large face rotations. Further, the regressor implicitly imposes a prior face shape on the solution, addressing occlusions and ambiguous face configurations. Its coarse-to-fine structure tackles the combinatorial explosion of parts deformation. In the experiments performed, 3DDE improves the state-of-the-art in 300W, COFW, AFLW and WFLW data sets. Finally, we perform cross-dataset experiments that reveal the existence of a significant data set bias in these benchmarks.