Robust Facial Landmark Localization Based on Texture and Pose Correlated Initialization

TL;DR

This paper introduces a robust initialization method for cascaded pose regression in facial landmark localization, combining texture and pose information to improve accuracy under occlusion and pose variations.

Contribution

The paper proposes RICPR, a novel initialization approach that integrates texture and pose correlation for enhanced facial landmark localization robustness.

Findings

Outperforms state-of-the-art methods on COFW dataset

Improves robustness to occlusion and pose variations

Achieves higher accuracy in landmark localization and occlusion detection

Abstract

Robust facial landmark localization remains a challenging task when faces are partially occluded. Recently, the cascaded pose regression has attracted increasing attentions, due to it's superior performance in facial landmark localization and occlusion detection. However, such an approach is sensitive to initialization, where an improper initialization can severly degrade the performance. In this paper, we propose a Robust Initialization for Cascaded Pose Regression (RICPR) by providing texture and pose correlated initial shapes for the testing face. By examining the correlation of local binary patterns histograms between the testing face and the training faces, the shapes of the training faces that are most correlated with the testing face are selected as the texture correlated initialization. To make the initialization more robust to various poses, we estimate the rough pose of the testing face according to five fiducial landmarks located by multitask cascaded convolutional networks. Then the pose correlated initial shapes are constructed by the mean face's shape and the rough testing face pose. Finally, the texture correlated and the pose correlated initial shapes are joined together as the robust initialization. We evaluate RICPR on the challenging dataset of COFW. The experimental results demonstrate that the proposed scheme achieves better performances than the state-of-the-art methods in facial landmark localization and occlusion detection.