Analyzing the Impact of Shape & Context on the Face Recognition Performance of Deep Networks

TL;DR

This paper investigates how 3D facial shape variations and contextual background influence deep face recognition performance, highlighting the importance of shape and context in training data for improved accuracy.

Contribution

It provides a detailed analysis of shape and context effects on face recognition, introducing new insights into data augmentation strategies involving 3D face rendering.

Findings

Facial shape significantly affects recognition accuracy.

Adding context and background improves model performance.

Shape perturbations cause notable identity distortions.

Abstract

In this article, we analyze how changing the underlying 3D shape of the base identity in face images can distort their overall appearance, especially from the perspective of deep face recognition. As done in popular training data augmentation schemes, we graphically render real and synthetic face images with randomly chosen or best-fitting 3D face models to generate novel views of the base identity. We compare deep features generated from these images to assess the perturbation these renderings introduce into the original identity. We perform this analysis at various degrees of facial yaw with the base identities varying in gender and ethnicity. Additionally, we investigate if adding some form of context and background pixels in these rendered images, when used as training data, further improves the downstream performance of a face recognition model. Our experiments demonstrate the significance of facial shape in accurate face matching and underpin the importance of contextual data for network training.