Towards End-to-End Neural Face Authentication in the Wild -- Quantifying and Compensating for Directional Lighting Effects

TL;DR

This paper investigates how directional lighting affects neural face recognition in the wild and demonstrates that fine-tuning can mitigate lighting effects, enabling robust on-device facial authentication without pre-processing.

Contribution

It introduces a synthetic relighting augmentation method and shows that fine-tuning neural models can compensate for directional lighting variations in facial recognition.

Findings

Top lighting has minimal impact on accuracy

Bottom lighting significantly affects recognition performance

Fine-tuning achieves robustness across lighting conditions

Abstract

The recent availability of low-power neural accelerator hardware, combined with improvements in end-to-end neural facial recognition algorithms provides, enabling technology for on-device facial authentication. The present research work examines the effects of directional lighting on a State-of-Art(SoA) neural face recognizer. A synthetic re-lighting technique is used to augment data samples due to the lack of public data-sets with sufficient directional lighting variations. Top lighting and its variants (top-left, top-right) are found to have minimal effect on accuracy, while bottom-left or bottom-right directional lighting has the most pronounced effects. Following the fine-tuning of network weights, the face recognition model is shown to achieve close to the original Receiver Operating Characteristic curve (ROC)performance across all lighting conditions and demonstrates an ability to generalize beyond the lighting augmentations used in the fine-tuning data-set. This work shows that an SoA neural face recognition model can be tuned to compensate for directional lighting effects, removing the need for a pre-processing step before applying facial recognition.