A Comprehensive Analysis of Deep Learning Based Representation for Face Recognition

TL;DR

This paper thoroughly evaluates deep learning face recognition models, VGG-Face and Lightened CNN, under various challenging conditions like pose, occlusion, illumination, and misalignment, highlighting the importance of preprocessing for optimal performance.

Contribution

It provides a comprehensive assessment of deep learning face representations under diverse real-world conditions, emphasizing the role of preprocessing techniques.

Findings

Deep learning models are robust to some misalignment and localization errors.

Preprocessing improves recognition accuracy under pose and illumination variations.

Performance drops when variations are not included in training datasets.

Abstract

Deep learning based approaches have been dominating the face recognition field due to the significant performance improvement they have provided on the challenging wild datasets. These approaches have been extensively tested on such unconstrained datasets, on the Labeled Faces in the Wild and YouTube Faces, to name a few. However, their capability to handle individual appearance variations caused by factors such as head pose, illumination, occlusion, and misalignment has not been thoroughly assessed till now. In this paper, we present a comprehensive study to evaluate the performance of deep learning based face representation under several conditions including the varying head pose angles, upper and lower face occlusion, changing illumination of different strengths, and misalignment due to erroneous facial feature localization. Two successful and publicly available deep learning models, namely VGG-Face and Lightened CNN have been utilized to extract face representations. The obtained results show that although deep learning provides a powerful representation for face recognition, it can still benefit from preprocessing, for example, for pose and illumination normalization in order to achieve better performance under various conditions. Particularly, if these variations are not included in the dataset used to train the deep learning model, the role of preprocessing becomes more crucial. Experimental results also show that deep learning based representation is robust to misalignment and can tolerate facial feature localization errors up to 10% of the interocular distance.