Variants of BERT, Random Forests and SVM approach for Multimodal Emotion-Target Sub-challenge

TL;DR

This paper explores multimodal emotion recognition using ensemble language models and traditional classifiers, achieving improved classification of topics, valence, and arousal in affective computing applications.

Contribution

It introduces a fusion approach combining ALBERT, RoBERTa, SVM, and Random Forests for emotion and topic classification in multimodal data.

Findings

Ensemble of ALBERT and RoBERTa improves topic classification.

SVM and Random Forests enhance valence and arousal prediction.

Feature selection contributes to performance improvement.

Abstract

Emotion recognition has become a major problem in computer vision in recent years that made a lot of effort by researchers to overcome the difficulties in this task. In the field of affective computing, emotion recognition has a wide range of applications, such as healthcare, robotics, human-computer interaction. Due to its practical importance for other tasks, many techniques and approaches have been investigated for different problems and various data sources. Nevertheless, comprehensive fusion of the audio-visual and language modalities to get the benefits from them is still a problem to solve. In this paper, we present and discuss our classification methodology for MuSe-Topic Sub-challenge, as well as the data and results. For the topic classification, we ensemble two language models which are ALBERT and RoBERTa to predict 10 classes of topics. Moreover, for the classification of valence and arousal, SVM and Random forests are employed in conjunction with feature selection to enhance the performance.