MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition

TL;DR

MER 2024 advances multimodal emotion recognition by expanding datasets, introducing open-vocabulary labeling, and promoting robustness and semi-supervised methods to better handle real-world complexities.

Contribution

This paper presents a new open-vocabulary emotion recognition track in the MER2024 competition, addressing annotation inaccuracies and encouraging flexible label generation.

Findings

Expanded dataset size for MER2024

Introduction of open-vocabulary emotion recognition

Baseline code provided for reproducibility

Abstract

Multimodal emotion recognition is an important research topic in artificial intelligence. Over the past few decades, researchers have made remarkable progress by increasing the dataset size and building more effective algorithms. However, due to problems such as complex environments and inaccurate annotations, current systems are hard to meet the demands of practical applications. Therefore, we organize the MER series of competitions to promote the development of this field. Last year, we launched MER2023, focusing on three interesting topics: multi-label learning, noise robustness, and semi-supervised learning. In this year's MER2024, besides expanding the dataset size, we further introduce a new track around open-vocabulary emotion recognition. The main purpose of this track is that existing datasets usually fix the label space and use majority voting to enhance the annotator consistency. However, this process may lead to inaccurate annotations, such as ignoring non-majority or non-candidate labels. In this track, we encourage participants to generate any number of labels in any category, aiming to describe emotional states as accurately as possible. Our baseline code relies on MERTools and is available at: https://github.com/zeroQiaoba/MERTools/tree/master/MER2024.