"I think you need help! Here's why": Understanding the Effect of Explanations on Automatic Facial Expression Recognition

TL;DR

This study investigates how explanations of facial expression recognition (FER) systems can enhance user trust, understanding, and performance in real-world human-computer interaction scenarios, demonstrating that explanations lead to better control and reduced errors.

Contribution

The paper provides empirical evidence that explanations of FER systems improve user trust, understanding, and task performance in practical applications.

Findings

Explanations increase user trust in FER systems.

Explanations improve user understanding of FER.

Explanations lead to fewer navigation collisions.

Abstract

Facial expression recognition (FER) has emerged as a promising approach to the development of emotion-aware intelligent systems. The performance of FER in multiple domains is continuously being improved, especially through advancements in data-driven learning approaches. However, a key challenge remains in utilizing FER in real-world contexts, namely ensuring user understanding of these systems and establishing a suitable level of user trust towards this technology. We conducted an empirical user study to investigate how explanations of FER can improve trust, understanding and performance in a human-computer interaction task that uses FER to trigger helpful hints during a navigation game. Our results showed that users provided with explanations of the FER system demonstrated improved control in using the system to their advantage, leading to a significant improvement in their understanding of the system, reduced collisions in the navigation game, as well as increased trust towards the system.