Multi-Task Learning for Emotion Descriptors Estimation at the fourth ABAW Challenge

TL;DR

This paper presents a multi-task learning framework that improves the estimation of facial affective descriptors, such as valence/arousal, expression, and action units, in wild conditions by leveraging task relations.

Contribution

The paper introduces a multi-task learning approach with feature sharing and label fusion to enhance performance on affective analysis tasks in unconstrained environments.

Findings

Improved accuracy on valence/arousal, expression, and action unit estimation.

Effective utilization of task relations through feature sharing.

Demonstrated robustness in wild, real-world conditions.

Abstract

Facial valence/arousal, expression and action unit are related tasks in facial affective analysis. However, the tasks only have limited performance in the wild due to the various collected conditions. The 4th competition on affective behavior analysis in the wild (ABAW) provided images with valence/arousal, expression and action unit labels. In this paper, we introduce multi-task learning framework to enhance the performance of three related tasks in the wild. Feature sharing and label fusion are used to utilize their relations. We conduct experiments on the provided training and validating data.