Exploiting Diverse Feature for Multimodal Sentiment Analysis

TL;DR

This paper presents a multimodal sentiment analysis approach that leverages diverse feature extraction and model ensemble techniques to predict continuous arousal and valence, achieving third place in the MuSe 2023 challenge.

Contribution

It introduces a novel combination of feature extraction methods and ensemble modeling to improve robustness in personalized sentiment prediction.

Findings

Achieved 0.8492 CCC for arousal

Achieved 0.8439 CCC for valence

Secured 3rd place in MuSe-Personalisation

Abstract

In this paper, we present our solution to the MuSe-Personalisation sub-challenge in the MuSe 2023 Multimodal Sentiment Analysis Challenge. The task of MuSe-Personalisation aims to predict the continuous arousal and valence values of a participant based on their audio-visual, language, and physiological signal modalities data. Considering different people have personal characteristics, the main challenge of this task is how to build robustness feature presentation for sentiment prediction. To address this issue, we propose exploiting diverse features. Specifically, we proposed a series of feature extraction methods to build a robust representation and model ensemble. We empirically evaluate the performance of the utilized method on the officially provided dataset. \textbf{As a result, we achieved 3rd place in the MuSe-Personalisation sub-challenge.} Specifically, we achieve the results of 0.8492 and 0.8439 for MuSe-Personalisation in terms of arousal and valence CCC.