Towards Large-Scale Pose-Invariant Face Recognition Using Face Defrontalization

TL;DR

This paper introduces face defrontalization, a novel method to improve pose-invariant face recognition by augmenting training data, demonstrating superior performance on large datasets without increasing inference time.

Contribution

The work proposes a new face defrontalization technique that enhances training data for face recognition models, outperforming existing frontalization methods on large-scale datasets.

Findings

Defrontalization improves recognition accuracy on large datasets.

The method outperforms state-of-the-art face frontalization on three datasets.

Limited improvement observed on small datasets like Multi-PIE.

Abstract

Face recognition under extreme head poses is a challenging task. Ideally, a face recognition system should perform well across different head poses, which is known as pose-invariant face recognition. To achieve pose invariance, current approaches rely on sophisticated methods, such as face frontalization and various facial feature extraction model architectures. However, these methods are somewhat impractical in real-life settings and are typically evaluated on small scientific datasets, such as Multi-PIE. In this work, we propose the inverse method of face frontalization, called face defrontalization, to augment the training dataset of facial feature extraction model. The method does not introduce any time overhead during the inference step. The method is composed of: 1) training an adapted face defrontalization FFWM model on a frontal-profile pairs dataset, which has been preprocessed using our proposed face alignment method; 2) training a ResNet-50 facial feature extraction model based on ArcFace loss on a raw and randomly defrontalized large-scale dataset, where defrontalization was performed with our previously trained face defrontalization model. Our method was compared with the existing approaches on four open-access datasets: LFW, AgeDB, CFP, and Multi-PIE. Defrontalization shows improved results compared to models without defrontalization, while the proposed adjustments show clear superiority over the state-of-the-art face frontalization FFWM method on three larger open-access datasets, but not on the small Multi-PIE dataset for extreme poses (75 and 90 degrees). The results suggest that at least some of the current methods may be overfitted to small datasets.