MTCAE-DFER: Multi-Task Cascaded Autoencoder for Dynamic Facial Expression Recognition

TL;DR

This paper introduces MTCAE-DFER, a multi-task cascaded autoencoder framework utilizing Vision Transformers to improve dynamic facial expression recognition by integrating local and global dynamic features and reducing overfitting.

Contribution

It proposes a novel plug-and-play cascaded decoder based on ViT architecture for multi-task learning in dynamic facial expression recognition, enhancing feature interaction and model robustness.

Findings

Outperforms state-of-the-art methods on public datasets.

Demonstrates improved generalization through multi-task learning.

Shows robustness and effectiveness of global-local feature interaction.

Abstract

This paper expands the cascaded network branch of the autoencoder-based multi-task learning (MTL) framework for dynamic facial expression recognition, namely Multi-Task Cascaded Autoencoder for Dynamic Facial Expression Recognition (MTCAE-DFER). MTCAE-DFER builds a plug-and-play cascaded decoder module, which is based on the Vision Transformer (ViT) architecture and employs the decoder concept of Transformer to reconstruct the multi-head attention module. The decoder output from the previous task serves as the query (Q), representing local dynamic features, while the Video Masked Autoencoder (VideoMAE) shared encoder output acts as both the key (K) and value (V), representing global dynamic features. This setup facilitates interaction between global and local dynamic features across related tasks. Additionally, this proposal aims to alleviate overfitting of complex large model. We utilize autoencoder-based multi-task cascaded learning approach to explore the impact of dynamic face detection and dynamic face landmark on dynamic facial expression recognition, which enhances the model's generalization ability. After we conduct extensive ablation experiments and comparison with state-of-the-art (SOTA) methods on various public datasets for dynamic facial expression recognition, the robustness of the MTCAE-DFER model and the effectiveness of global-local dynamic feature interaction among related tasks have been proven.