2D-3D Attention and Entropy for Pose Robust 2D Facial Recognition

TL;DR

This paper introduces a novel domain adaptive framework that improves pose-invariant 2D facial recognition by leveraging joint attention and entropy regularization to better correlate 2D images with 3D facial data, enhancing recognition accuracy across large pose variations.

Contribution

The paper presents a new framework combining joint attention mapping and entropy regularization to improve pose robustness in 2D facial recognition by leveraging 3D data correlations.

Findings

Outperforms existing methods on FaceScape and ARL-VTF datasets.

Achieves at least 7.1% and 1.57% improvements in TAR @ 1% FAR for profile poses.

Effectively enhances pose invariance in facial recognition systems.

Abstract

Despite recent advances in facial recognition, there remains a fundamental issue concerning degradations in performance due to substantial perspective (pose) differences between enrollment and query (probe) imagery. Therefore, we propose a novel domain adaptive framework to facilitate improved performances across large discrepancies in pose by enabling image-based (2D) representations to infer properties of inherently pose invariant point cloud (3D) representations. Specifically, our proposed framework achieves better pose invariance by using (1) a shared (joint) attention mapping to emphasize common patterns that are most correlated between 2D facial images and 3D facial data and (2) a joint entropy regularizing loss to promote better consistency$\unicode{x2014}$enhancing correlations among the intersecting 2D and 3D representations$\unicode{x2014}$by leveraging both attention maps. This framework is evaluated on FaceScape and ARL-VTF datasets, where it outperforms competitive methods by achieving profile (90$\unicode{x00b0}$$\unicode{x002b}$) TAR @ 1$\unicode{x0025}$ FAR improvements of at least 7.1$\unicode{x0025}$ and 1.57$\unicode{x0025}$, respectively.