Meta Transfer Learning for Emotion Recognition

TL;DR

This paper introduces a PathNet-based transfer learning approach for emotion recognition that effectively transfers emotional knowledge across different visual and audio domains, improving accuracy over traditional fine-tuning methods.

Contribution

It proposes a novel PathNet-based transfer learning method that preserves and transfers emotional knowledge across multiple domains, enhancing emotion recognition performance.

Findings

Outperforms recent fine-tuning transfer learning methods

Improves emotion recognition accuracy on multiple datasets

Demonstrates robustness across facial and speech emotion recognition tasks

Abstract

Deep learning has been widely adopted in automatic emotion recognition and has lead to significant progress in the field. However, due to insufficient annotated emotion datasets, pre-trained models are limited in their generalization capability and thus lead to poor performance on novel test sets. To mitigate this challenge, transfer learning performing fine-tuning on pre-trained models has been applied. However, the fine-tuned knowledge may overwrite and/or discard important knowledge learned from pre-trained models. In this paper, we address this issue by proposing a PathNet-based transfer learning method that is able to transfer emotional knowledge learned from one visual/audio emotion domain to another visual/audio emotion domain, and transfer the emotional knowledge learned from multiple audio emotion domains into one another to improve overall emotion recognition accuracy. To show the robustness of our proposed system, various sets of experiments for facial expression recognition and speech emotion recognition task on three emotion datasets: SAVEE, EMODB, and eNTERFACE have been carried out. The experimental results indicate that our proposed system is capable of improving the performance of emotion recognition, making its performance substantially superior to the recent proposed fine-tuning/pre-trained models based transfer learning methods.