Neonatal Face and Facial Landmark Detection from Video Recordings

TL;DR

This study develops and evaluates a YOLOv7Face-based model for automated neonatal face and facial landmark detection from video recordings, achieving high accuracy and aiding neonatal health assessments.

Contribution

It introduces a transfer learning approach with data augmentation and model fusion to improve neonatal face and landmark detection accuracy.

Findings

Achieved 84.8% mean average precision for face detection

Attained 0.072 normalized mean error for landmark detection

Outperformed existing methods in neonatal face analysis

Abstract

This paper explores automated face and facial landmark detection of neonates, which is an important first step in many video-based neonatal health applications, such as vital sign estimation, pain assessment, sleep-wake classification, and jaundice detection. Utilising three publicly available datasets of neonates in the clinical environment, 366 images (258 subjects) and 89 (66 subjects) were annotated for training and testing, respectively. Transfer learning was applied to two YOLO-based models, with input training images augmented with random horizontal flipping, photo-metric colour distortion, translation and scaling during each training epoch. Additionally, the re-orientation of input images and fusion of trained deep learning models was explored. Our proposed model based on YOLOv7Face outperformed existing methods with a mean average precision of 84.8% for face detection, and a normalised mean error of 0.072 for facial landmark detection. Overall, this will assist in the development of fully automated neonatal health assessment algorithms.