A survey and classification of face alignment methods based on face models

TL;DR

This paper provides a comprehensive survey of face alignment methods focusing on various face models, including their training, interpretation, and application, highlighting the preference for 3D models in extreme poses and deep learning approaches.

Contribution

It offers the first detailed review of different face models used in face alignment, including their training and fitting techniques, and discusses future research directions.

Findings

3D face models are preferred for extreme poses

Deep learning methods often utilize heatmaps

The survey covers face model training and fitting techniques

Abstract

A face model is a mathematical representation of the distinct features of a human face. Traditionally, face models were built using a set of fiducial points or landmarks, each point ideally located on a facial feature, i.e., corner of the eye, tip of the nose, etc. Face alignment is the process of fitting the landmarks in a face model to the respective ground truth positions in an input image containing a face. Despite significant research on face alignment in the past decades, no review analyses various face models used in the literature. Catering to three types of readers - beginners, practitioners and researchers in face alignment, we provide a comprehensive analysis of different face models used for face alignment. We include the interpretation and training of the face models along with the examples of fitting the face model to a new face image. We found that 3D-based face models are preferred in cases of extreme face pose, whereas deep learning-based methods often use heatmaps. Moreover, we discuss the possible future directions of face models in the field of face alignment.