MuSe 2020 -- The First International Multimodal Sentiment Analysis in Real-life Media Challenge and Workshop

TL;DR

MuSe 2020 is a pioneering multimodal sentiment analysis challenge focusing on real-life media, integrating audio-visual and language data to recognize emotions, topics, and trustworthiness with new datasets and baseline models.

Contribution

It introduces the MuSe-CaR in-the-wild database and establishes benchmark tasks and baseline models for multimodal sentiment and trustworthiness analysis.

Findings

Achieved a CCC of .2568 for emotion prediction

Reached 76.78% accuracy on topic recognition

Obtained a CCC of .4359 for trustworthiness prediction

Abstract

Multimodal Sentiment Analysis in Real-life Media (MuSe) 2020 is a Challenge-based Workshop focusing on the tasks of sentiment recognition, as well as emotion-target engagement and trustworthiness detection by means of more comprehensively integrating the audio-visual and language modalities. The purpose of MuSe 2020 is to bring together communities from different disciplines; mainly, the audio-visual emotion recognition community (signal-based), and the sentiment analysis community (symbol-based). We present three distinct sub-challenges: MuSe-Wild, which focuses on continuous emotion (arousal and valence) prediction; MuSe-Topic, in which participants recognise domain-specific topics as the target of 3-class (low, medium, high) emotions; and MuSe-Trust, in which the novel aspect of trustworthiness is to be predicted. In this paper, we provide detailed information on MuSe-CaR, the first of its kind in-the-wild database, which is utilised for the challenge, as well as the state-of-the-art features and modelling approaches applied. For each sub-challenge, a competitive baseline for participants is set; namely, on test we report for MuSe-Wild a combined (valence and arousal) CCC of .2568, for MuSe-Topic a score (computed as 0.34$\cdot$ UAR + 0.66$\cdot$F1) of 76.78 % on the 10-class topic and 40.64 % on the 3-class emotion prediction, and for MuSe-Trust a CCC of .4359.