Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition

TL;DR

This paper introduces DAN, a novel facial expression recognition network that leverages multi-head cross attention to focus on multiple facial regions simultaneously, improving recognition accuracy by capturing subtle and high-order local features.

Contribution

The paper proposes a new multi-component network architecture combining feature clustering, multi-head attention, and attention fusion for improved facial expression recognition.

Findings

Achieves state-of-the-art results on AffectNet, RAF-DB, and SFEW 2.0 datasets.

Demonstrates the effectiveness of multi-head attention in capturing diverse facial regions.

Shows robustness of the proposed method across multiple public datasets.

Abstract

We present a novel facial expression recognition network, called Distract your Attention Network (DAN). Our method is based on two key observations. Firstly, multiple classes share inherently similar underlying facial appearance, and their differences could be subtle. Secondly, facial expressions exhibit themselves through multiple facial regions simultaneously, and the recognition requires a holistic approach by encoding high-order interactions among local features. To address these issues, we propose our DAN with three key components: Feature Clustering Network (FCN), Multi-head cross Attention Network (MAN), and Attention Fusion Network (AFN). The FCN extracts robust features by adopting a large-margin learning objective to maximize class separability. In addition, the MAN instantiates a number of attention heads to simultaneously attend to multiple facial areas and build attention maps on these regions. Further, the AFN distracts these attentions to multiple locations before fusing the attention maps to a comprehensive one. Extensive experiments on three public datasets (including AffectNet, RAF-DB, and SFEW 2.0) verified that the proposed method consistently achieves state-of-the-art facial expression recognition performance. Code will be made available at https://github.com/yaoing/DAN.