Detecting facial landmarks in the video based on a hybrid framework

TL;DR

This paper introduces a hybrid detection-tracking-detection framework for real-time facial landmark detection in videos, combining traditional detection, deep learning, and tracking to improve efficiency and accuracy.

Contribution

The paper proposes a novel hybrid framework that enhances facial landmark detection in videos by integrating detection, tracking, and re-detection, reducing computational cost.

Findings

More effective facial landmark detection in videos.

Lower processing time compared to frame-by-frame methods.

Improved accuracy through validation of face bounding boxes.

Abstract

To dynamically detect the facial landmarks in the video, we propose a novel hybrid framework termed as detection-tracking-detection (DTD). First, the face bounding box is achieved from the first frame of the video sequence based on a traditional face detection method. Then, a landmark detector detects the facial landmarks, which is based on a cascaded deep convolution neural network (DCNN). Next, the face bounding box in the current frame is estimated and validated after the facial landmarks in the previous frame are tracked based on the median flow. Finally, the facial landmarks in the current frame are exactly detected from the validated face bounding box via the landmark detector. Experimental results indicate that the proposed framework can detect the facial landmarks in the video sequence more effectively and with lower consuming time compared to the frame-by-frame method via the DCNN.