Facial Feature Tracking under Varying Facial Expressions and Face Poses based on Restricted Boltzmann Machines

TL;DR

This paper presents a novel face shape prior model based on Restricted Boltzmann Machines that improves facial feature tracking robustness across varying expressions and poses.

Contribution

It introduces a deep belief network-based face shape model and a 3-way RBM to handle expression and pose variations, respectively.

Findings

Robust facial feature point tracking under expression and pose variations

Effective face shape modeling using RBMs and deep belief networks

Improved accuracy demonstrated on benchmark databases

Abstract

Facial feature tracking is an active area in computer vision due to its relevance to many applications. It is a nontrivial task, since faces may have varying facial expressions, poses or occlusions. In this paper, we address this problem by proposing a face shape prior model that is constructed based on the Restricted Boltzmann Machines (RBM) and their variants. Specifically, we first construct a model based on Deep Belief Networks to capture the face shape variations due to varying facial expressions for near-frontal view. To handle pose variations, the frontal face shape prior model is incorporated into a 3-way RBM model that could capture the relationship between frontal face shapes and non-frontal face shapes. Finally, we introduce methods to systematically combine the face shape prior models with image measurements of facial feature points. Experiments on benchmark databases show that with the proposed method, facial feature points can be tracked robustly and accurately even if faces have significant facial expressions and poses.