Temporal Label Hierachical Network for Compound Emotion Recognition

TL;DR

This paper presents a hierarchical temporal network for recognizing complex compound emotions, utilizing pre-trained models, a time pyramid structure, and fine-grained labels to improve emotion recognition accuracy.

Contribution

It introduces a novel hierarchical temporal network with a time pyramid structure and fine-grained label utilization for improved compound emotion recognition.

Findings

Effective in recognizing complex emotions in practical scenarios

Utilizes pre-trained ResNet18 and Transformer models

Constructs a coarse-to-fine classification framework

Abstract

The emotion recognition has attracted more attention in recent decades. Although significant progress has been made in the recognition technology of the seven basic emotions, existing methods are still hard to tackle compound emotion recognition that occurred commonly in practical application. This article introduces our achievements in the 7th Field Emotion Behavior Analysis (ABAW) competition. In the competition, we selected pre trained ResNet18 and Transformer, which have been widely validated, as the basic network framework. Considering the continuity of emotions over time, we propose a time pyramid structure network for frame level emotion prediction. Furthermore. At the same time, in order to address the lack of data in composite emotion recognition, we utilized fine-grained labels from the DFEW database to construct training data for emotion categories in competitions. Taking into account the characteristics of valence arousal of various complex emotions, we constructed a classification framework from coarse to fine in the label space.