Fine-Grained Head Pose Estimation Without Keypoints

TL;DR

This paper introduces a robust, keypoint-free method for head pose estimation using a multi-loss CNN trained on synthetic data, achieving state-of-the-art results without relying on landmark detection.

Contribution

It proposes a novel direct pose estimation approach with a multi-loss CNN trained on synthetic data, bypassing landmark detection and head models.

Findings

Achieves state-of-the-art results on in-the-wild pose benchmarks.

Performs well on depth-based pose datasets, closing the gap with depth methods.

Open-sources code and pre-trained models for reproducibility.

Abstract

Estimating the head pose of a person is a crucial problem that has a large amount of applications such as aiding in gaze estimation, modeling attention, fitting 3D models to video and performing face alignment. Traditionally head pose is computed by estimating some keypoints from the target face and solving the 2D to 3D correspondence problem with a mean human head model. We argue that this is a fragile method because it relies entirely on landmark detection performance, the extraneous head model and an ad-hoc fitting step. We present an elegant and robust way to determine pose by training a multi-loss convolutional neural network on 300W-LP, a large synthetically expanded dataset, to predict intrinsic Euler angles (yaw, pitch and roll) directly from image intensities through joint binned pose classification and regression. We present empirical tests on common in-the-wild pose benchmark datasets which show state-of-the-art results. Additionally we test our method on a dataset usually used for pose estimation using depth and start to close the gap with state-of-the-art depth pose methods. We open-source our training and testing code as well as release our pre-trained models.