Affective Behavior Analysis using Task-adaptive and AU-assisted Graph Network

TL;DR

This paper proposes a multi-task learning framework for affective behavior analysis in-the-wild, integrating large pre-trained models, task-adaptive feature extraction, and AU-assisted graph networks to improve performance across three tasks.

Contribution

It introduces a novel combination of Dinov2, task-adaptive blocks, and AU-GCN for multi-task affective behavior analysis, advancing the state-of-the-art in this domain.

Findings

Achieved an evaluation score of 1.2542 on the validation set.

Effectively utilized AU correlations to enhance expression and valence-arousal tasks.

Demonstrated the effectiveness of task-adaptive feature extraction.

Abstract

In this paper, we present our solution and experiment result for the Multi-Task Learning Challenge of the 7th Affective Behavior Analysis in-the-wild(ABAW7) Competition. This challenge consists of three tasks: action unit detection, facial expression recognition, and valance-arousal estimation. We address the research problems of this challenge from three aspects: 1)For learning robust visual feature representations, we introduce the pre-trained large model Dinov2. 2) To adaptively extract the required features of eack task, we design a task-adaptive block that performs cross-attention between a set of learnable query vectors and pre-extracted features. 3) By proposing the AU-assisted Graph Convolutional Network(AU-GCN), we make full use of the correlation information between AUs to assist in solving the EXPR and VA tasks. Finally, we achieve the evaluation measure of \textbf{1.2542} on the validation set provided by the organizers.