Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition

TL;DR

This paper presents a multi-task streaming network for affective recognition that leverages prior facial expression embeddings and intrinsic relationships among emotion representations, demonstrating improved accuracy on the Aff-Wild2 dataset.

Contribution

The paper introduces a novel multi-task streaming network that incorporates prior facial expression embeddings and models the intrinsic association among multiple emotion representations.

Findings

Effective multi-task streaming network for affective recognition.

Leveraging prior facial expression embeddings improves recognition accuracy.

Ablation studies confirm the effectiveness of the proposed approach.

Abstract

Automatic affective recognition has been an important research topic in human computer interaction (HCI) area. With recent development of deep learning techniques and large scale in-the-wild annotated datasets, the facial emotion analysis is now aimed at challenges in the real world settings. In this paper, we introduce our submission to the 2nd Affective Behavior Analysis in-the-wild (ABAW2) Competition. In dealing with different emotion representations, including Categorical Emotions (CE), Action Units (AU), and Valence Arousal (VA), we propose a multi-task streaming network by a heuristic that the three representations are intrinsically associated with each other. Besides, we leverage an advanced facial expression embedding as prior knowledge, which is capable of capturing identity-invariant expression features while preserving the expression similarities, to aid the down-streaming recognition tasks. The extensive quantitative evaluations as well as ablation studies on the Aff-Wild2 dataset prove the effectiveness of our proposed prior aided streaming network approach.