Fun Selfie Filters in Face Recognition: Impact Assessment and Removal

TL;DR

This paper examines how fun selfie filters impact face recognition accuracy and introduces a GAN-based removal method that significantly improves biometric performance after filter application.

Contribution

It provides a comprehensive analysis of selfie filter effects on face recognition and proposes a novel GAN-based technique for filter removal to enhance recognition performance.

Findings

Selfie filters can significantly reduce face recognition accuracy.

Filters covering key facial features cause the most disruption.

GAN-based removal improves biometric system performance after filter application.

Abstract

This work investigates the impact of fun selfie filters, which are frequently used to modify selfies, on face recognition systems. Based on a qualitative assessment and classification of freely available mobile applications, ten relevant fun selfie filters are selected to create a database. To this end, the selected filters are automatically applied to face images of public face image databases. Different state-of-the-art methods are used to evaluate the influence of fun selfie filters on the performance of face detection using dlib, RetinaFace, and a COTS method, sample quality estimated by FaceQNet and MagFace, and recognition accuracy employing ArcFace and a COTS algorithm. The obtained results indicate that selfie filters negatively affect face recognition modules, especially if fun selfie filters cover a large region of the face, where the mouth, nose, and eyes are covered. To mitigate such unwanted effects, a GAN-based selfie filter removal algorithm is proposed which consists of a segmentation module, a perceptual network, and a generation module. In a cross-database experiment the application of the presented selfie filter removal technique has shown to significantly improve the biometric performance of the underlying face recognition systems.