Robust Facial Landmark Detection under Significant Head Poses and Occlusion

TL;DR

This paper introduces a unified cascade regression framework that robustly detects facial landmarks in images with severe occlusion and large head poses, outperforming existing methods in challenging scenarios.

Contribution

A novel supervised regression approach for occlusion estimation and a unified framework handling both occlusion and pose variations in facial landmark detection.

Findings

Significantly better performance on occluded and profile face images.

Comparable results on general in-the-wild images.

Effective iterative update of occlusion and landmark positions.

Abstract

There have been tremendous improvements for facial landmark detection on general "in-the-wild" images. However, it is still challenging to detect the facial landmarks on images with severe occlusion and images with large head poses (e.g. profile face). In fact, the existing algorithms usually can only handle one of them. In this work, we propose a unified robust cascade regression framework that can handle both images with severe occlusion and images with large head poses. Specifically, the method iteratively predicts the landmark occlusions and the landmark locations. For occlusion estimation, instead of directly predicting the binary occlusion vectors, we introduce a supervised regression method that gradually updates the landmark visibility probabilities in each iteration to achieve robustness. In addition, we explicitly add occlusion pattern as a constraint to improve the performance of occlusion prediction. For landmark detection, we combine the landmark visibility probabilities, the local appearances, and the local shapes to iteratively update their positions. The experimental results show that the proposed method is significantly better than state-of-the-art works on images with severe occlusion and images with large head poses. It is also comparable to other methods on general "in-the-wild" images.