POSEidon: Face-from-Depth for Driver Pose Estimation

TL;DR

This paper introduces POSEidon, a deep learning framework that accurately estimates driver head pose from depth images in real time, improving robustness under challenging conditions like occlusions and lighting changes.

Contribution

The work presents a novel neural network architecture for head pose estimation from depth images and a Face-from-Depth approach to enhance face understanding, outperforming existing methods.

Findings

Outperforms state-of-the-art head pose estimation methods

Operates in real-time at over 30 fps

Effective on multiple challenging datasets

Abstract

Fast and accurate upper-body and head pose estimation is a key task for automatic monitoring of driver attention, a challenging context characterized by severe illumination changes, occlusions and extreme poses. In this work, we present a new deep learning framework for head localization and pose estimation on depth images. The core of the proposal is a regression neural network, called POSEidon, which is composed of three independent convolutional nets followed by a fusion layer, specially conceived for understanding the pose by depth. In addition, to recover the intrinsic value of face appearance for understanding head position and orientation, we propose a new Face-from-Depth approach for learning image faces from depth. Results in face reconstruction are qualitatively impressive. We test the proposed framework on two public datasets, namely Biwi Kinect Head Pose and ICT-3DHP, and on Pandora, a new challenging dataset mainly inspired by the automotive setup. Results show that our method overcomes all recent state-of-art works, running in real time at more than 30 frames per second.