A Framework for Real-Time Face and Facial Feature Tracking using Optical Flow Pre-estimation and Template Tracking

TL;DR

This paper introduces a real-time framework combining optical flow and template tracking to robustly track head and facial movements, effectively handling fast motions and partial occlusions.

Contribution

The novel integration of optical flow and template tracking with occlusion handling and a switching scheme enhances robustness in real-time facial feature tracking.

Findings

Robust real-time head and facial feature tracking achieved.

Effective handling of fast movements and motion-blur.

Reduced impact of partial occlusions on tracking accuracy.

Abstract

This work presents a framework for tracking head movements and capturing the movements of the mouth and both the eyebrows in real-time. We present a head tracker which is a combination of a optical flow and a template based tracker. The estimation of the optical flow head tracker is used as starting point for the template tracker which fine-tunes the head estimation. This approach together with re-updating the optical flow points prevents the head tracker from drifting. This combination together with our switching scheme, makes our tracker very robust against fast movement and motion-blur. We also propose a way to reduce the influence of partial occlusion of the head. In both the optical flow and the template based tracker we identify and exclude occluded points.