Two-Aspect Information Fusion Model For ABAW4 Multi-task Challenge

TL;DR

This paper introduces a novel end-to-end multi-task learning model that effectively integrates various emotion descriptors for improved prediction in the ABAW4 challenge.

Contribution

It presents a new architecture that fully integrates different emotion-related information types, addressing the lack of interaction modeling in prior approaches.

Findings

The proposed model outperforms existing methods on ABAW4 tasks.

Full integration of emotion descriptors improves prediction accuracy.

Experimental results validate the effectiveness of the approach.

Abstract

In this paper, we propose the solution to the Multi-Task Learning (MTL) Challenge of the 4th Affective Behavior Analysis in-the-wild (ABAW) competition. The task of ABAW is to predict frame-level emotion descriptors from videos: discrete emotional state; valence and arousal; and action units. Although researchers have proposed several approaches and achieved promising results in ABAW, current works in this task rarely consider interactions between different emotion descriptors. To this end, we propose a novel end to end architecture to achieve full integration of different types of information. Experimental results demonstrate the effectiveness of our proposed solution.