3D Face Alignment Through Fusion of Head Pose Information and Features

TL;DR

This paper introduces a novel 3D face alignment method that fuses head pose information with feature maps, utilizing a dual-dimensional network and knowledge distillation to enhance accuracy and robustness in facial landmark prediction.

Contribution

The study proposes a new face alignment approach that integrates head pose data into a dual-dimensional network and employs knowledge distillation for dense landmark prediction, improving performance over existing methods.

Findings

Effective fusion of head pose and feature maps improves alignment accuracy.

The method outperforms state-of-the-art on AFLW2000-3D, AFLW, and BIWI datasets.

Robust face alignment achieved through dual-dimensional network structure.

Abstract

The ability of humans to infer head poses from face shapes, and vice versa, indicates a strong correlation between the two. Accordingly, recent studies on face alignment have employed head pose information to predict facial landmarks in computer vision tasks. In this study, we propose a novel method that employs head pose information to improve face alignment performance by fusing said information with the feature maps of a face alignment network, rather than simply using it to initialize facial landmarks. Furthermore, the proposed network structure performs robust face alignment through a dual-dimensional network using multidimensional features represented by 2D feature maps and a 3D heatmap. For effective dense face alignment, we also propose a prediction method for facial geometric landmarks through training based on knowledge distillation using predicted keypoints. We experimentally assessed the correlation between the predicted facial landmarks and head pose information, as well as variations in the accuracy of facial landmarks with respect to the quality of head pose information. In addition, we demonstrated the effectiveness of the proposed method through a competitive performance comparison with state-of-the-art methods on the AFLW2000-3D, AFLW, and BIWI datasets.