Learning Emotional-Blinded Face Representations

TL;DR

This paper introduces two novel face representations that obscure emotional expressions to enhance privacy and fairness, while maintaining performance in identity and demographic recognition tasks.

Contribution

It presents two methods for learning emotion-blinded face features, addressing privacy concerns and demonstrating their effectiveness across multiple face-related tasks.

Findings

Emotion information can be eliminated without significantly affecting verification and demographic tasks.

The proposed methods improve fairness in attractiveness classification by reducing emotional bias.

Face representations retain high performance in identity, gender, and ethnicity recognition.

Abstract

We propose two face representations that are blind to facial expressions associated to emotional responses. This work is in part motivated by new international regulations for personal data protection, which enforce data controllers to protect any kind of sensitive information involved in automatic processes. The advances in Affective Computing have contributed to improve human-machine interfaces but, at the same time, the capacity to monitorize emotional responses triggers potential risks for humans, both in terms of fairness and privacy. We propose two different methods to learn these expression-blinded facial features. We show that it is possible to eliminate information related to emotion recognition tasks, while the performance of subject verification, gender recognition, and ethnicity classification are just slightly affected. We also present an application to train fairer classifiers in a case study of attractiveness classification with respect to a protected facial expression attribute. The results demonstrate that it is possible to reduce emotional information in the face representation while retaining competitive performance in other face-based artificial intelligence tasks.