Real-Time Shape Tracking of Facial Landmarks
Hyungjoon Kim, Hyeonwoo Kim, and Eenjun Hwang

TL;DR
This paper presents a real-time facial landmark shape tracking method that combines deep learning and point tracking to improve accuracy and robustness in virtual makeup applications.
Contribution
It introduces a hybrid approach using SegNet for precise landmark detection and KLT for tracking, addressing point loss and computational efficiency issues.
Findings
Accurate real-time shape tracking demonstrated in experiments.
Effective combination reduces point loss problems.
Maintains real-time performance in virtual makeup scenarios.
Abstract
Detection of facial landmarks and accurate tracking of their shape are essential in real-time virtual makeup applications, where users can see the makeups effect by moving their face in different directions. Typical face tracking techniques detect diverse facial landmarks and track them using a point tracker such as the Kanade-Lucas-Tomasi (KLT) point tracker. Typically, 5 or 64 points are used for tracking a face. Even though these points are sufficient to track the approximate locations of facial landmarks, they are not sufficient to track the exact shape of facial landmarks. In this paper, we propose a method that can track the exact shape of facial landmarks in real-time by combining a deep learning technique and a point tracker. We detect facial landmarks accurately using SegNet, which performs semantic segmentation based on deep learning. Edge points of detected landmarks are…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Video Surveillance and Tracking Methods · 3D Shape Modeling and Analysis
MethodsConvolution · Kaiming Initialization · Batch Normalization · *Communicated@Fast*How Do I Communicate to Expedia? · Max Pooling · Softmax · SegNet
