Active Learning Based Fine-Tuning Framework for Speech Emotion Recognition

TL;DR

This paper introduces an active learning-based fine-tuning framework for speech emotion recognition that improves accuracy and reduces training time by selectively using informative samples and minimizing the information gap through task adaptation pre-training.

Contribution

It combines task adaptation pre-training with active learning to enhance SER performance and efficiency, addressing limitations of existing methods.

Findings

20% sample usage yields 8.45% accuracy improvement

Reduces fine-tuning time by 79%

Effective in large-scale noisy data scenarios

Abstract

Speech emotion recognition (SER) has drawn increasing attention for its applications in human-machine interaction. However, existing SER methods ignore the information gap between the pre-training speech recognition task and the downstream SER task, leading to sub-optimal performance. Moreover, they require much time to fine-tune on each specific speech dataset, restricting their effectiveness in real-world scenes with large-scale noisy data. To address these issues, we propose an active learning (AL) based Fine-Tuning framework for SER that leverages task adaptation pre-training (TAPT) and AL methods to enhance performance and efficiency. Specifically, we first use TAPT to minimize the information gap between the pre-training and the downstream task. Then, AL methods are used to iteratively select a subset of the most informative and diverse samples for fine-tuning, reducing time consumption. Experiments demonstrate that using only 20\%pt. samples improves 8.45\%pt. accuracy and reduces 79\%pt. time consumption.