Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild

TL;DR

This paper introduces an intensity-aware loss and a global convolution-attention block to improve dynamic facial expression recognition in videos, especially handling varying expression intensities in real-world scenarios.

Contribution

The paper proposes a novel intensity-aware loss and a global convolution-attention block to enhance DFER performance under varying expression intensities.

Findings

Outperforms state-of-the-art DFER methods on DFEW and FERV39k datasets.

Effectively distinguishes low-intensity expressions in videos.

Improves intra-class feature separation for dynamic facial expressions.

Abstract

Compared with the image-based static facial expression recognition (SFER) task, the dynamic facial expression recognition (DFER) task based on video sequences is closer to the natural expression recognition scene. However, DFER is often more challenging. One of the main reasons is that video sequences often contain frames with different expression intensities, especially for the facial expressions in the real-world scenarios, while the images in SFER frequently present uniform and high expression intensities. However, if the expressions with different intensities are treated equally, the features learned by the networks will have large intra-class and small inter-class differences, which is harmful to DFER. To tackle this problem, we propose the global convolution-attention block (GCA) to rescale the channels of the feature maps. In addition, we introduce the intensity-aware loss (IAL) in the training process to help the network distinguish the samples with relatively low expression intensities. Experiments on two in-the-wild dynamic facial expression datasets (i.e., DFEW and FERV39k) indicate that our method outperforms the state-of-the-art DFER approaches. The source code will be made publicly available.