Multimodal Emotion Recognition for One-Minute-Gradual Emotion Challenge

TL;DR

This paper presents a multimodal approach combining acoustic, visual, and textual features with SVM fusion to predict continuous arousal and valence emotions in the One-Minute-Gradual Emotion Challenge, significantly outperforming baselines.

Contribution

The work introduces a multimodal fusion method using SVMs for emotion prediction in a continuous emotion recognition challenge, achieving state-of-the-art results.

Findings

CCC scores of 0.397 for arousal and 0.520 for valence on validation data.

Outperforms baseline CCC scores of 0.15 and 0.23.

Effective multimodal feature integration enhances emotion prediction accuracy.

Abstract

The continuous dimensional emotion modelled by arousal and valence can depict complex changes of emotions. In this paper, we present our works on arousal and valence predictions for One-Minute-Gradual (OMG) Emotion Challenge. Multimodal representations are first extracted from videos using a variety of acoustic, video and textual models and support vector machine (SVM) is then used for fusion of multimodal signals to make final predictions. Our solution achieves Concordant Correlation Coefficient (CCC) scores of 0.397 and 0.520 on arousal and valence respectively for the validation dataset, which outperforms the baseline systems with the best CCC scores of 0.15 and 0.23 on arousal and valence by a large margin.