I am Only Happy When There is Light: The Impact of Environmental Changes on Affective Facial Expressions Recognition

TL;DR

This paper investigates how environmental factors like lighting affect the accuracy of facial expression recognition in human-robot interaction, highlighting the importance of robustness in real-world applications.

Contribution

It provides an analysis of the impact of environmental changes on deep learning-based facial expression recognition in HRI scenarios, emphasizing robustness considerations.

Findings

Recognition accuracy varies significantly with lighting changes.

Small environmental variations can lead to large interpretative differences.

Robustness to environmental changes is crucial for real-world HRI applications.

Abstract

Human-robot interaction (HRI) benefits greatly from advances in the machine learning field as it allows researchers to employ high-performance models for perceptual tasks like detection and recognition. Especially deep learning models, either pre-trained for feature extraction or used for classification, are now established methods to characterize human behaviors in HRI scenarios and to have social robots that understand better those behaviors. As HRI experiments are usually small-scale and constrained to particular lab environments, the questions are how well can deep learning models generalize to specific interaction scenarios, and further, how good is their robustness towards environmental changes? These questions are important to address if the HRI field wishes to put social robotic companions into real environments acting consistently, i.e. changing lighting conditions or moving people should still produce the same recognition results. In this paper, we study the impact of different image conditions on the recognition of arousal and valence from human facial expressions using the FaceChannel framework \cite{Barro20}. Our results show how the interpretation of human affective states can differ greatly in either the positive or negative direction even when changing only slightly the image properties. We conclude the paper with important points to consider when employing deep learning models to ensure sound interpretation of HRI experiments.