Disentangling 3D Pose in A Dendritic CNN for Unconstrained 2D Face Alignment

TL;DR

This paper introduces PCD-CNN, a novel face landmark localization method that explicitly disentangles 3D pose using a Bayesian approach, improving accuracy across various face poses without increasing network complexity.

Contribution

The paper proposes a pose-conditioned dendritic CNN that explicitly disentangles 3D pose, reducing localization error and extending applicability to different datasets with variable landmarks.

Findings

Reduces localization error by up to 15%

Effective on challenging datasets like AFLW, AFW, COFW, IBUG

Achieves accurate landmark localization across extreme and medium face poses

Abstract

Heatmap regression has been used for landmark localization for quite a while now. Most of the methods use a very deep stack of bottleneck modules for heatmap classification stage, followed by heatmap regression to extract the keypoints. In this paper, we present a single dendritic CNN, termed as Pose Conditioned Dendritic Convolution Neural Network (PCD-CNN), where a classification network is followed by a second and modular classification network, trained in an end to end fashion to obtain accurate landmark points. Following a Bayesian formulation, we disentangle the 3D pose of a face image explicitly by conditioning the landmark estimation on pose, making it different from multi-tasking approaches. Extensive experimentation shows that conditioning on pose reduces the localization error by making it agnostic to face pose. The proposed model can be extended to yield variable number of landmark points and hence broadening its applicability to other datasets. Instead of increasing depth or width of the network, we train the CNN efficiently with Mask-Softmax Loss and hard sample mining to achieve upto $15\%$ reduction in error compared to state-of-the-art methods for extreme and medium pose face images from challenging datasets including AFLW, AFW, COFW and IBUG.