More Is Better: A MoE-Based Emotion Recognition Framework with Human Preference Alignment

TL;DR

This paper introduces a MoE-based emotion recognition system that leverages multiple input modalities, unlabeled data, and human preference alignment techniques to improve accuracy and robustness in semi-supervised settings.

Contribution

It presents a novel semi-supervised emotion recognition framework using a Mixture of Experts, integrating diverse modalities, consensus pseudo-labeling, and ensemble voting for better performance.

Findings

Achieved an F1-score of 0.8772 on MER2025-SEMI dataset.

Ranked 2nd in the MER2025 challenge.

Demonstrated effectiveness of multi-modal experts and consensus pseudo-labeling.

Abstract

In this paper, we present our solution for the semi-supervised learning track (MER-SEMI) in MER2025. We propose a comprehensive framework, grounded in the principle that "more is better," to construct a robust Mixture of Experts (MoE) emotion recognition system. Our approach integrates a diverse range of input modalities as independent experts, including novel signals such as knowledge from large Vision-Language Models (VLMs) and temporal Action Unit (AU) information. To effectively utilize unlabeled data, we introduce a consensus-based pseudo-labeling strategy, generating high-quality labels from the agreement between a baseline model and Gemini, which are then used in a two-stage training paradigm. Finally, we employ a multi-expert voting ensemble combined with a rule-based re-ranking process to correct prediction bias and better align the outputs with human preferences. Evaluated on the MER2025-SEMI challenge dataset, our method achieves an F1-score of 0.8772 on the test set, ranking 2nd in the track. Our code is available at https://github.com/zhuyjan/MER2025-MRAC25.