Fast Facial Landmark Detection and Applications: A Survey

TL;DR

This survey reviews state-of-the-art neural network-based facial landmark detection methods, emphasizing in-the-wild datasets, inference speed, and practical applications, to guide future research in robust and efficient face analysis.

Contribution

It categorizes recent algorithms, compares datasets, and discusses inference speed and vulnerabilities, providing a comprehensive overview for advancing facial landmark detection.

Findings

Neural networks significantly improve in-the-wild facial landmark detection.

Comparison of datasets highlights challenges in pose and occlusion.

Inference speed analysis aids deployment on mobile devices.

Abstract

Dense facial landmark detection is one of the key elements of face processing pipeline. It is used in virtual face reenactment, emotion recognition, driver status tracking, etc. Early approaches were suitable for facial landmark detection in controlled environments only, which is clearly insufficient. Neural networks have shown an astonishing qualitative improvement for in-the-wild face landmark detection problem, and are now being studied by many researchers in the field. Numerous bright ideas are proposed, often complimentary to each other. However, exploration of the whole volume of novel approaches is quite challenging. Therefore, we present this survey, where we summarize state-of-the-art algorithms into categories, provide a comparison of recently introduced in-the-wild datasets (e.g., 300W, AFLW, COFW, WFLW) that contain images with large pose, face occlusion, taken in unconstrained conditions. In addition to quality, applications require fast inference, and preferably on mobile devices. Hence, we include information about algorithm inference speed both on desktop and mobile hardware, which is rarely studied. Importantly, we highlight problems of algorithms, their applications, vulnerabilities, and briefly touch on established methods. We hope that the reader will find many novel ideas, will see how the algorithms are used in applications, which will enable further research.