Track Facial Points in Unconstrained Videos

TL;DR

This paper introduces an incremental learning method for real-time, person-specific facial point tracking in unconstrained videos, improving accuracy and efficiency over existing offline models.

Contribution

It proposes a unified framework that updates face alignment models on the fly using part-based representation, cascade regression, and deep neural network evaluation to prevent drift.

Findings

Outperforms existing methods in accuracy and efficiency

Effective in unconstrained, wild conditions

Validated on both image and video datasets

Abstract

Tracking Facial Points in unconstrained videos is challenging due to the non-rigid deformation that changes over time. In this paper, we propose to exploit incremental learning for person-specific alignment in wild conditions. Our approach takes advantage of part-based representation and cascade regression for robust and efficient alignment on each frame. Unlike existing methods that usually rely on models trained offline, we incrementally update the representation subspace and the cascade of regressors in a unified framework to achieve personalized modeling on the fly. To alleviate the drifting issue, the fitting results are evaluated using a deep neural network, where well-aligned faces are picked out to incrementally update the representation and fitting models. Both image and video datasets are employed to valid the proposed method. The results demonstrate the superior performance of our approach compared with existing approaches in terms of fitting accuracy and efficiency.