MOS: A Low Latency and Lightweight Framework for Face Detection, Landmark Localization, and Head Pose Estimation

TL;DR

This paper introduces a low latency, lightweight multi-task network that simultaneously performs face detection, landmark localization, and head pose estimation, optimized for resource-constrained environments like robots.

Contribution

It proposes a novel multi-task learning framework with pose and uncertainty losses, and introduces online feedback sampling to improve performance on small and hard faces.

Findings

Achieves state-of-the-art results on WIDER FACE, AFLW, and AFLW2000 datasets.

Operates efficiently on low-resource devices like ARM-based systems.

Improves detection and estimation accuracy for challenging face angles.

Abstract

With the emergence of service robots and surveillance cameras, dynamic face recognition (DFR) in wild has received much attention in recent years. Face detection and head pose estimation are two important steps for DFR. Very often, the pose is estimated after the face detection. However, such sequential computations lead to higher latency. In this paper, we propose a low latency and lightweight network for simultaneous face detection, landmark localization and head pose estimation. Inspired by the observation that it is more challenging to locate the facial landmarks for faces with large angles, a pose loss is proposed to constrain the learning. Moreover, we also propose an uncertainty multi-task loss to learn the weights of individual tasks automatically. Another challenge is that robots often use low computational units like ARM based computing core and we often need to use lightweight networks instead of the heavy ones, which lead to performance drop especially for small and hard faces. In this paper, we propose online feedback sampling to augment the training samples across different scales, which increases the diversity of training data automatically. Through validation in commonly used WIDER FACE, AFLW and AFLW2000 datasets, the results show that the proposed method achieves the state-of-the-art performance in low computational resources. The code and data will be available at https://github.com/lyp-deeplearning/MOS-Multi-Task-Face-Detect.