Simultaneous Facial Landmark Detection, Pose and Deformation Estimation under Facial Occlusion

TL;DR

This paper introduces a unified, robust framework for simultaneously detecting facial landmarks, estimating head pose, and analyzing facial deformation, effectively handling facial occlusions through an iterative cascade approach.

Contribution

A novel integrated model that jointly performs facial landmark detection, pose estimation, and deformation analysis, improving robustness to occlusion compared to separate methods.

Findings

Effective on benchmark datasets with occlusion

Accurate joint estimation of landmarks, pose, and deformation

Outperforms existing independent methods

Abstract

Facial landmark detection, head pose estimation, and facial deformation analysis are typical facial behavior analysis tasks in computer vision. The existing methods usually perform each task independently and sequentially, ignoring their interactions. To tackle this problem, we propose a unified framework for simultaneous facial landmark detection, head pose estimation, and facial deformation analysis, and the proposed model is robust to facial occlusion. Following a cascade procedure augmented with model-based head pose estimation, we iteratively update the facial landmark locations, facial occlusion, head pose and facial de- formation until convergence. The experimental results on benchmark databases demonstrate the effectiveness of the proposed method for simultaneous facial landmark detection, head pose and facial deformation estimation, even if the images are under facial occlusion.