A Real-Time Multi-Task Learning System for Joint Detection of Face, Facial Landmark and Head Pose

TL;DR

This paper presents a real-time multi-task system based on YOLOv8 that jointly detects faces, facial landmarks, and head poses, effectively handling large-angle face poses with high efficiency.

Contribution

It introduces a multi-task detection framework extending YOLOv8 with additional landmark regression, optimized for real-time joint face, landmark, and head pose detection.

Findings

Effective large-angle face pose handling

Real-time performance across tasks

Validated on 300W-LP and AFLW2000-3D datasets

Abstract

Extreme head postures pose a common challenge across a spectrum of facial analysis tasks, including face detection, facial landmark detection (FLD), and head pose estimation (HPE). These tasks are interdependent, where accurate FLD relies on robust face detection, and HPE is intricately associated with these key points. This paper focuses on the integration of these tasks, particularly when addressing the complexities posed by large-angle face poses. The primary contribution of this study is the proposal of a real-time multi-task detection system capable of simultaneously performing joint detection of faces, facial landmarks, and head poses. This system builds upon the widely adopted YOLOv8 detection framework. It extends the original object detection head by incorporating additional landmark regression head, enabling efficient localization of crucial facial landmarks. Furthermore, we conduct optimizations and enhancements on various modules within the original YOLOv8 framework. To validate the effectiveness and real-time performance of our proposed model, we conduct extensive experiments on 300W-LP and AFLW2000-3D datasets. The results obtained verify the capability of our model to tackle large-angle face pose challenges while delivering real-time performance across these interconnected tasks.