To Frontalize or Not To Frontalize: Do We Really Need Elaborate Pre-processing To Improve Face Recognition?

TL;DR

This paper investigates whether elaborate pre-processing like face frontalization improves face recognition, introducing a new automatic frontalization method and evaluating its effectiveness against existing techniques.

Contribution

The paper introduces a novel automatic single-image frontalization scheme and evaluates its impact on face recognition performance compared to other pose correction methods.

Findings

Frontalization can enhance face recognition accuracy.

The new frontalization method outperforms existing algorithms.

Pose normalization effects vary depending on the dataset.

Abstract

Face recognition performance has improved remarkably in the last decade. Much of this success can be attributed to the development of deep learning techniques such as convolutional neural networks (CNNs). While CNNs have pushed the state-of-the-art forward, their training process requires a large amount of clean and correctly labelled training data. If a CNN is intended to tolerate facial pose, then we face an important question: should this training data be diverse in its pose distribution, or should face images be normalized to a single pose in a pre-processing step? To address this question, we evaluate a number of popular facial landmarking and pose correction algorithms to understand their effect on facial recognition performance. Additionally, we introduce a new, automatic, single-image frontalization scheme that exceeds the performance of current algorithms. CNNs trained using sets of different pre-processing methods are used to extract features from the Point and Shoot Challenge (PaSC) and CMU Multi-PIE datasets. We assert that the subsequent verification and recognition performance serves to quantify the effectiveness of each pose correction scheme.