Multi-spectral Facial Landmark Detection

TL;DR

This paper introduces a neural network-based method called Deep Multi-Spectral Learning (DMSL) for detecting facial landmarks in thermal and visible images, addressing challenges like occlusion and different face orientations.

Contribution

The paper presents a novel dual-submodel neural network architecture capable of accurately detecting facial landmarks in thermal images, a less-explored area in face analysis.

Findings

DMSL outperforms existing methods on thermal face datasets.

The model is robust to occlusion and varied face orientations.

A new annotated thermal face dataset with landmarks is provided.

Abstract

Thermal face image analysis is favorable for certain circumstances. For example, illumination-sensitive applications, like nighttime surveillance; and privacy-preserving demanded access control. However, the inadequate study on thermal face image analysis calls for attention in responding to the industry requirements. Detecting facial landmark points are important for many face analysis tasks, such as face recognition, 3D face reconstruction, and face expression recognition. In this paper, we propose a robust neural network enabled facial landmark detection, namely Deep Multi-Spectral Learning (DMSL). Briefly, DMSL consists of two sub-models, i.e. face boundary detection, and landmark coordinates detection. Such an architecture demonstrates the capability of detecting the facial landmarks on both visible and thermal images. Particularly, the proposed DMSL model is robust in facial landmark detection where the face is partially occluded, or facing different directions. The experiment conducted on Eurecom's visible and thermal paired database shows the superior performance of DMSL over the state-of-the-art for thermal facial landmark detection. In addition to that, we have annotated a thermal face dataset with their respective facial landmark for the purpose of experimentation.