A Comprehensive Evaluation Framework for the Study of the Effects of Facial Filters on Face Recognition Accuracy

TL;DR

This paper presents a comprehensive framework for evaluating how various facial filters affect automated face recognition accuracy across different social media platforms and cultures, enabling large-scale analysis and potential performance improvements.

Contribution

It introduces a scalable framework with a diverse dataset and filter selection process to study the impact of facial filters on recognition accuracy, addressing limitations of prior small-scale studies.

Findings

Filters from different platforms have varying impacts on recognition accuracy.

Cross-cultural differences influence filter effects on face recognition.

Filtering effects can be detected and mitigated in face embedding space.

Abstract

Facial filters are now commonplace for social media users around the world. Previous work has demonstrated that facial filters can negatively impact automated face recognition performance. However, these studies focus on small numbers of hand-picked filters in particular styles. In order to more effectively incorporate the wide ranges of filters present on various social media applications, we introduce a framework that allows for larger-scale study of the impact of facial filters on automated recognition. This framework includes a controlled dataset of face images, a principled filter selection process that selects a representative range of filters for experimentation, and a set of experiments to evaluate the filters' impact on recognition. We demonstrate our framework with a case study of filters from the American applications Instagram and Snapchat and the Chinese applications Meitu and Pitu to uncover cross-cultural differences. Finally, we show how the filtering effect in a face embedding space can easily be detected and restored to improve face recognition performance.