Lessons Learnt: Revisit Key Training Strategies for Effective Speech Emotion Recognition in the Wild

TL;DR

This paper revisits and optimizes key training strategies for speech emotion recognition in naturalistic conditions, demonstrating that simple modifications can significantly improve model robustness and performance.

Contribution

The study identifies effective training strategies like balancing, activation functions, and fine-tuning that enhance SER performance without increasing model complexity.

Findings

Achieved a valence CCC of 0.6953 with a multi-modal fusion model.

Fine-tuning RoBERTa and WavLM separately improves valence performance.

Focal loss and activation functions boost performance without added complexity.

Abstract

In this study, we revisit key training strategies in machine learning often overlooked in favor of deeper architectures. Specifically, we explore balancing strategies, activation functions, and fine-tuning techniques to enhance speech emotion recognition (SER) in naturalistic conditions. Our findings show that simple modifications improve generalization with minimal architectural changes. Our multi-modal fusion model, integrating these optimizations, achieves a valence CCC of 0.6953, the best valence score in Task 2: Emotional Attribute Regression. Notably, fine-tuning RoBERTa and WavLM separately in a single-modality setting, followed by feature fusion without training the backbone extractor, yields the highest valence performance. Additionally, focal loss and activation functions significantly enhance performance without increasing complexity. These results suggest that refining core components, rather than deepening models, leads to more robust SER in-the-wild.