How Should Voice Assistants Deal With Users' Emotions?

TL;DR

This study investigates how humans respond to avatars displaying negative emotions, aiming to inform how voice assistants should react to users' emotions by analyzing user reactions in a controlled experiment.

Contribution

It provides insights into human emotional responses to avatars, highlighting gender differences, to guide the development of emotion-aware voice assistants.

Findings

Users mainly reacted with neutral emotion.

Gender differences influence emotional reactions.

Results inform design of emotion-aware interfaces.

Abstract

There is a growing body of research in HCI on detecting the users' emotions. Once it is possible to detect users' emotions reliably, the next question is how an emotion-aware interface should react to the detected emotion. In a first step, we tried to find out how humans deal with the negative emotions of an avatar. The hope behind this approach was to identify human strategies, which we can then mimic in an emotion-aware voice assistant. We present a user study in which participants were confronted with an angry, sad, or frightened avatar. Their task was to make the avatar happy by talking to it. We recorded the voice signal and analyzed it. The results show that users predominantly reacted with neutral emotion. However, we also found gender differences, which opens a range of questions.