EmoRL: Continuous Acoustic Emotion Classification using Deep Reinforcement Learning

TL;DR

EmoRL is a deep reinforcement learning-based model for real-time acoustic emotion classification that predicts emotions incrementally, reducing latency and eliminating the need for utterance segmentation in human-robot interaction scenarios.

Contribution

It introduces a reinforcement learning approach for continuous emotion detection, enabling early prediction without relying on utterance boundaries.

Findings

Achieves comparable accuracy to state-of-the-art models.

Enables earlier emotion prediction during speech.

Reduces latency in emotion detection.

Abstract

Acoustically expressed emotions can make communication with a robot more efficient. Detecting emotions like anger could provide a clue for the robot indicating unsafe/undesired situations. Recently, several deep neural network-based models have been proposed which establish new state-of-the-art results in affective state evaluation. These models typically start processing at the end of each utterance, which not only requires a mechanism to detect the end of an utterance but also makes it difficult to use them in a real-time communication scenario, e.g. human-robot interaction. We propose the EmoRL model that triggers an emotion classification as soon as it gains enough confidence while listening to a person speaking. As a result, we minimize the need for segmenting the audio signal for classification and achieve lower latency as the audio signal is processed incrementally. The method is competitive with the accuracy of a strong baseline model, while allowing much earlier prediction.