Multitask Multi-database Emotion Recognition

TL;DR

This paper presents a unified deep learning approach for emotion recognition from facial expressions and valence-arousal estimation across multiple databases, utilizing knowledge distillation and multi-task learning to improve performance.

Contribution

The authors introduce a multi-task, multi-database emotion recognition model that employs knowledge distillation and task correlation exploitation, advancing multi-database emotion analysis.

Findings

Achieved promising validation results on the AffWild2 database.

Effectively used knowledge distillation to handle incomplete labels.

Enhanced performance by exploiting inter-task correlations and shared videos.

Abstract

In this work, we introduce our submission to the 2nd Affective Behavior Analysis in-the-wild (ABAW) 2021 competition. We train a unified deep learning model on multi-databases to perform two tasks: seven basic facial expressions prediction and valence-arousal estimation. Since these databases do not contains labels for all the two tasks, we have applied the distillation knowledge technique to train two networks: one teacher and one student model. The student model will be trained using both ground truth labels and soft labels derived from the pretrained teacher model. During the training, we add one more task, which is the combination of the two mentioned tasks, for better exploiting inter-task correlations. We also exploit the sharing videos between the two tasks of the AffWild2 database that is used in the competition, to further improve the performance of the network. Experiment results shows that the network have achieved promising results on the validation set of the AffWild2 database. Code and pretrained model are publicly available at https://github.com/glmanhtu/multitask-abaw-2021